[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:10:08 dist-test-slave-wftq krb5kdc[10167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:10:08 dist-test-slave-wftq krb5kdc[10167](info): set up 2 sockets
Sep 29 19:10:08 dist-test-slave-wftq krb5kdc[10167](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260929 19:10:09.481652 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:10:09 dist-test-slave-wftq krb5kdc[10167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:10:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:10:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:10:09.605050 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:43713
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:44965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:43713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:09.684538 10180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:09.684772 10180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:09.684794 10180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:09.685949 10180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:09.685993 10180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:09.686007 10180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:09.686017 10180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:09.686028 10180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:09.687490 10180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:44965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:43713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:43713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:09.687685 10180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:09.687899 10180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:09.693441 10186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:09.693400 10185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:09.693399 10188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:09.693418 10180 server_base.cc:1034] running on GCE node
I20260929 19:10:09.693956 10180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:09.694248 10180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:09.695407 10180 hybrid_clock.cc:648] HybridClock initialized: now 1790709009695390 us; error 38 us; skew 500 ppm
Sep 29 19:10:09 dist-test-slave-wftq krb5kdc[10167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709009, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:09.698161 10180 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:10:09.698624 10180 webserver.cc:460] Webserver started at http://127.9.236.62:44013/ using document root <none> and password file <none>
I20260929 19:10:09.698835 10180 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:09.698891 10180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:09.699018 10180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:09.700078 10180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "a4acb023a0bf4a16ae09f5a1bf90ff37"
format_stamp: "Formatted at 2026-09-29 19:10:09 on dist-test-slave-wftq"
server_key: "180764799ee2ab3547f776237c15d3d7"
server_key_iv: "9c025bfd7bf294fce61e303bd644029c"
server_key_version: "encryptionkey@0"
I20260929 19:10:09.700415 10180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "a4acb023a0bf4a16ae09f5a1bf90ff37"
format_stamp: "Formatted at 2026-09-29 19:10:09 on dist-test-slave-wftq"
server_key: "180764799ee2ab3547f776237c15d3d7"
server_key_iv: "9c025bfd7bf294fce61e303bd644029c"
server_key_version: "encryptionkey@0"
I20260929 19:10:09.701813 10180 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:10:09.702634 10195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:09.702847 10180 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:09.702952 10180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "a4acb023a0bf4a16ae09f5a1bf90ff37"
format_stamp: "Formatted at 2026-09-29 19:10:09 on dist-test-slave-wftq"
server_key: "180764799ee2ab3547f776237c15d3d7"
server_key_iv: "9c025bfd7bf294fce61e303bd644029c"
server_key_version: "encryptionkey@0"
I20260929 19:10:09.703013 10180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:09.716193 10180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:09.718091 10180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:09.718223 10180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:09.722314 10180 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:43713
I20260929 19:10:09.722373 10247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:43713 every 8 connection(s)
I20260929 19:10:09.722749 10180 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:10:09.723738 10248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:09.726393 10248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37: Bootstrap starting.
I20260929 19:10:09.727028 10248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:09.727309 10248 log.cc:824] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:09.728199 10248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37: No bootstrap required, opened a new log
I20260929 19:10:09.730187 10248 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4acb023a0bf4a16ae09f5a1bf90ff37" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43713 } }
I20260929 19:10:09.730329 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 10180
I20260929 19:10:09.730363 10248 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:09.730392 10248 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4acb023a0bf4a16ae09f5a1bf90ff37, State: Initialized, Role: FOLLOWER
I20260929 19:10:09.730424 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:10:09.730541 10248 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4acb023a0bf4a16ae09f5a1bf90ff37" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43713 } }
I20260929 19:10:09.730628 10248 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:10:09.730664 10248 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:10:09.730724 10248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:09.731153 10160 external_mini_cluster.cc:1428] Setting key 322d4e53b4c8811f6ddd5c09563ff9fd
I20260929 19:10:09.731364 10248 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4acb023a0bf4a16ae09f5a1bf90ff37" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43713 } }
I20260929 19:10:09.731472 10248 leader_election.cc:302] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4acb023a0bf4a16ae09f5a1bf90ff37; no voters:
I20260929 19:10:09.731660 10248 leader_election.cc:288] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:10:09.731695 10251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:09.731842 10251 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37 [term 1 LEADER]: Becoming Leader. State: Replica: a4acb023a0bf4a16ae09f5a1bf90ff37, State: Running, Role: LEADER
I20260929 19:10:09.731935 10251 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4acb023a0bf4a16ae09f5a1bf90ff37" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43713 } }
I20260929 19:10:09.731969 10248 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:09.733528 10252 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4acb023a0bf4a16ae09f5a1bf90ff37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4acb023a0bf4a16ae09f5a1bf90ff37" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43713 } } }
I20260929 19:10:09.733661 10252 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:09.733786 10253 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4acb023a0bf4a16ae09f5a1bf90ff37. Latest consensus state: current_term: 1 leader_uuid: "a4acb023a0bf4a16ae09f5a1bf90ff37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4acb023a0bf4a16ae09f5a1bf90ff37" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43713 } } }
I20260929 19:10:09.734159 10253 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37 [sys.catalog]: This master's current role is: LEADER
W20260929 19:10:09.734597 10267 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:10:09.734658 10267 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:10:09.734776 10269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:09.734953 10269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:10:09.737226 10269 catalog_manager.cc:1347] Generated new cluster ID: 00f81041269f498db71b8b38dd77e293
I20260929 19:10:09.737287 10269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:10:09 dist-test-slave-wftq krb5kdc[10167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:09.748225 10269 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:10:09.748760 10269 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:09.750630 10255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:09.732492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49578 (local address 127.9.236.62:43713)
0929 19:10:09.732724 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:09.732734 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:09.732785 (+ 51us) server_negotiation.cc:407] Connection header received
0929 19:10:09.734825 (+ 2040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:09.734835 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:09.734898 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:09.735107 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:09.737603 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:09.739083 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:09.740722 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:09.741413 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:09.741551 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:09.746092 (+ 4541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:09.746097 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:09.746097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:09.746111 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:09.748072 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:09.748676 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:09.748679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:09.748679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:09.748784 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:09.749138 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:09.749139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:09.749140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:09.749395 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:09.749512 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:09.750260 (+ 748us) server_negotiation.cc:299] Negotiation successful
0929 19:10:09.750404 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":88,"threads_started":1}
I20260929 19:10:09.753275 10269 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4acb023a0bf4a16ae09f5a1bf90ff37: Generated new TSK 0
I20260929 19:10:09.753496 10269 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.236.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
I20260929 19:10:09.780169 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:43713
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:44965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:09.862769 10275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:09.862995 10275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:09.863027 10275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:09.864871 10275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:09.864923 10275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:09.864961 10275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:10:09.867156 10275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:44965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:43713
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:09.867544 10275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:09.867799 10275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:09.873392 10280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:09.873461 10283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:09.873420 10281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:09.873975 10275 server_base.cc:1034] running on GCE node
I20260929 19:10:09.874233 10275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:09.874490 10275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:09.875679 10275 hybrid_clock.cc:648] HybridClock initialized: now 1790709009875666 us; error 36 us; skew 500 ppm
Sep 29 19:10:09 dist-test-slave-wftq krb5kdc[10167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709009, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:09.878331 10275 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:10:09.878888 10275 webserver.cc:460] Webserver started at http://127.9.236.1:44399/ using document root <none> and password file <none>
I20260929 19:10:09.879161 10275 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:09.879227 10275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:09.879360 10275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:09.880787 10275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "041674841c5a43fdab84e89c8df3cd19"
format_stamp: "Formatted at 2026-09-29 19:10:09 on dist-test-slave-wftq"
server_key: "84bcfbf2f7be1c02bdbb32700734a1b4"
server_key_iv: "d31304fdd3d3029c82eb6a2c8c87c18e"
server_key_version: "encryptionkey@0"
I20260929 19:10:09.881165 10275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "041674841c5a43fdab84e89c8df3cd19"
format_stamp: "Formatted at 2026-09-29 19:10:09 on dist-test-slave-wftq"
server_key: "84bcfbf2f7be1c02bdbb32700734a1b4"
server_key_iv: "d31304fdd3d3029c82eb6a2c8c87c18e"
server_key_version: "encryptionkey@0"
I20260929 19:10:09.883054 10275 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260929 19:10:09.883936 10290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:09.884127 10275 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:09.884261 10275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "041674841c5a43fdab84e89c8df3cd19"
format_stamp: "Formatted at 2026-09-29 19:10:09 on dist-test-slave-wftq"
server_key: "84bcfbf2f7be1c02bdbb32700734a1b4"
server_key_iv: "d31304fdd3d3029c82eb6a2c8c87c18e"
server_key_version: "encryptionkey@0"
I20260929 19:10:09.884369 10275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:09.901860 10275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:09.904536 10275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:09.904695 10275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:09.905004 10275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:09.905465 10275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:09.905514 10275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:09.905611 10275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:09.905642 10275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:09.913396 10275 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:37683
I20260929 19:10:09.913497 10403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:37683 every 8 connection(s)
I20260929 19:10:09.913805 10275 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:10:09.915740 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 10275
I20260929 19:10:09.915853 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:10:09.916018 10160 external_mini_cluster.cc:1428] Setting key ae96d1d8dd9436289791185a2d1e8b9e
Sep 29 19:10:09 dist-test-slave-wftq krb5kdc[10167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709009, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.236.2@KRBTEST.COM; defaulting to no policy
I20260929 19:10:09.924134 10255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:09.914600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:37597 (local address 127.9.236.62:43713)
0929 19:10:09.914664 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:09.914666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:09.914886 (+ 220us) server_negotiation.cc:407] Connection header received
0929 19:10:09.915408 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:09.915409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:09.915421 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:09.915471 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:09.916731 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:09.917374 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:09.918587 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:09.918903 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:09.918982 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:09.921443 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:09.921444 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:09.921445 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:09.921453 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:09.922792 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:09.923191 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:09.923192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:09.923193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:09.923253 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:09.923476 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:09.923477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:09.923477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:09.923593 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:09.923649 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:09.923934 (+ 285us) server_negotiation.cc:299] Negotiation successful
0929 19:10:09.924030 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 19:10:09.924319 10406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:09.914532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:43713 (local address 127.9.236.1:37597)
0929 19:10:09.914756 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:09.914797 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:09.915312 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:09.915540 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:09.915545 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:09.916307 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:09.916568 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:09.916573 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:09.917468 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:09.917469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:09.918499 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:09.918501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:09.918987 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:09.918988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:09.919113 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:09.919639 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:09.919649 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:09.921348 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:09.922873 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:09.922880 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:09.922883 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:09.923124 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:09.923308 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:09.923309 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:09.923309 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:09.923422 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:09.923645 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:09.923651 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:09.923813 (+ 162us) client_negotiation.cc:769] Sending connection context
0929 19:10:09.923907 (+ 94us) client_negotiation.cc:240] Negotiation successful
0929 19:10:09.924089 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":94,"threads_started":1}
I20260929 19:10:09.924837 10404 heartbeater.cc:347] Connected to a master server at 127.9.236.62:43713
I20260929 19:10:09.924937 10404 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:09.925169 10404 heartbeater.cc:511] Master 127.9.236.62:43713 requested a full tablet report, sending...
I20260929 19:10:09.925690 10212 ts_manager.cc:194] Registered new tserver with Master: 041674841c5a43fdab84e89c8df3cd19 (127.9.236.1:37683)
I20260929 19:10:09.926373 10212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:37597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
I20260929 19:10:09.940487 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:43713
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:44965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:10.019040 10409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:10.019255 10409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:10.019279 10409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:10.020365 10409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:10.020406 10409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:10.020430 10409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:10:10.021826 10409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:44965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:43713
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:10.022008 10409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:10.022190 10409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:10.027024 10415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:10.027163 10409 server_base.cc:1034] running on GCE node
W20260929 19:10:10.027024 10417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:10.027043 10414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:10.027474 10409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:10.027714 10409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:10.028884 10409 hybrid_clock.cc:648] HybridClock initialized: now 1790709010028862 us; error 32 us; skew 500 ppm
Sep 29 19:10:10 dist-test-slave-wftq krb5kdc[10167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709010, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:10.031543 10409 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:10:10.031993 10409 webserver.cc:460] Webserver started at http://127.9.236.2:36659/ using document root <none> and password file <none>
I20260929 19:10:10.032188 10409 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:10.032233 10409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:10.032375 10409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:10.033530 10409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "8ea1ab4eacb444c29e1ed1a5d81518b7"
format_stamp: "Formatted at 2026-09-29 19:10:10 on dist-test-slave-wftq"
server_key: "11a5c5fba0c1feb2a6cf7045ce59a954"
server_key_iv: "d15adf0c9ae75e9a354f968890e5802d"
server_key_version: "encryptionkey@0"
I20260929 19:10:10.033876 10409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "8ea1ab4eacb444c29e1ed1a5d81518b7"
format_stamp: "Formatted at 2026-09-29 19:10:10 on dist-test-slave-wftq"
server_key: "11a5c5fba0c1feb2a6cf7045ce59a954"
server_key_iv: "d15adf0c9ae75e9a354f968890e5802d"
server_key_version: "encryptionkey@0"
I20260929 19:10:10.035214 10409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:10.035977 10424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:10.036247 10409 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:10.036338 10409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "8ea1ab4eacb444c29e1ed1a5d81518b7"
format_stamp: "Formatted at 2026-09-29 19:10:10 on dist-test-slave-wftq"
server_key: "11a5c5fba0c1feb2a6cf7045ce59a954"
server_key_iv: "d15adf0c9ae75e9a354f968890e5802d"
server_key_version: "encryptionkey@0"
I20260929 19:10:10.036406 10409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:10.056152 10409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:10.058024 10409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:10.058151 10409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:10.058403 10409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:10.058728 10409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:10.058763 10409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:10.058794 10409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:10.058809 10409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:10.065239 10409 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:40915
I20260929 19:10:10.065376 10537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:40915 every 8 connection(s)
I20260929 19:10:10.065634 10409 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
Sep 29 19:10:10 dist-test-slave-wftq krb5kdc[10167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709010, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:10.075382 10255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:10.066519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:34761 (local address 127.9.236.62:43713)
0929 19:10:10.066608 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:10.066611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:10.066792 (+ 181us) server_negotiation.cc:407] Connection header received
0929 19:10:10.067436 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:10.067437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:10.067451 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:10.067502 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:10.068602 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:10.069191 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:10.070216 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:10.070423 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:10.070482 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:10.072791 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:10.072792 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:10.072793 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:10.072801 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:10.074115 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:10.074491 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:10.074492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:10.074493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:10.074544 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:10.074757 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:10.074758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:10.074759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:10.074875 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:10.074931 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:10.075193 (+ 262us) server_negotiation.cc:299] Negotiation successful
0929 19:10:10.075275 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 19:10:10.075592 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:10.066448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:43713 (local address 127.9.236.2:34761)
0929 19:10:10.066626 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:10.066677 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:10.067321 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:10.067593 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:10.067597 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:10.068249 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:10.068501 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:10.068506 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:10.069272 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:10.069273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:10.070131 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:10.070134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:10.070493 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:10.070494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:10.070624 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:10.071202 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:10.071211 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:10.072701 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:10.074199 (+ 1498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:10.074205 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:10.074208 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:10.074426 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:10.074599 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:10.074600 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:10.074600 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:10.074705 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:10.074929 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:10.074935 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:10.075076 (+ 141us) client_negotiation.cc:769] Sending connection context
0929 19:10:10.075167 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 19:10:10.075353 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260929 19:10:10.075812 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 10409
I20260929 19:10:10.075919 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:10:10.076085 10160 external_mini_cluster.cc:1428] Setting key 3b8fefd18aebd4988ce55a6fe473837e
I20260929 19:10:10.076138 10538 heartbeater.cc:347] Connected to a master server at 127.9.236.62:43713
I20260929 19:10:10.076251 10538 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:10.076457 10538 heartbeater.cc:511] Master 127.9.236.62:43713 requested a full tablet report, sending...
I20260929 19:10:10.076956 10212 ts_manager.cc:194] Registered new tserver with Master: 8ea1ab4eacb444c29e1ed1a5d81518b7 (127.9.236.2:40915)
I20260929 19:10:10.077397 10212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:34761
WARNING: no policy specified for kudu/127.9.236.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
I20260929 19:10:10.100813 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:43713
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:44965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:10.179443 10543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:10.179656 10543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:10.179679 10543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:10.180749 10543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:10.180797 10543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:10.180821 10543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:10:10.182199 10543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:44965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:43713
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:10.182385 10543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:10.182555 10543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:10.187433 10548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:10.187438 10549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:10.187475 10551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:10.187674 10543 server_base.cc:1034] running on GCE node
I20260929 19:10:10.187896 10543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:10.188155 10543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:10.189345 10543 hybrid_clock.cc:648] HybridClock initialized: now 1790709010189319 us; error 52 us; skew 500 ppm
Sep 29 19:10:10 dist-test-slave-wftq krb5kdc[10167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709010, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:10.192121 10543 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:10:10.192623 10543 webserver.cc:460] Webserver started at http://127.9.236.3:44087/ using document root <none> and password file <none>
I20260929 19:10:10.192878 10543 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:10.192929 10543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:10.193060 10543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:10.194300 10543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "678f33cc3d034716adb04c180f5c9072"
format_stamp: "Formatted at 2026-09-29 19:10:10 on dist-test-slave-wftq"
server_key: "a8efc367693616df2a567c1a0452996f"
server_key_iv: "25cde261d36c4541919d88ad4b64c631"
server_key_version: "encryptionkey@0"
I20260929 19:10:10.194646 10543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "678f33cc3d034716adb04c180f5c9072"
format_stamp: "Formatted at 2026-09-29 19:10:10 on dist-test-slave-wftq"
server_key: "a8efc367693616df2a567c1a0452996f"
server_key_iv: "25cde261d36c4541919d88ad4b64c631"
server_key_version: "encryptionkey@0"
I20260929 19:10:10.196043 10543 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:10:10.196775 10558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:10.196970 10543 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:10:10.197063 10543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "678f33cc3d034716adb04c180f5c9072"
format_stamp: "Formatted at 2026-09-29 19:10:10 on dist-test-slave-wftq"
server_key: "a8efc367693616df2a567c1a0452996f"
server_key_iv: "25cde261d36c4541919d88ad4b64c631"
server_key_version: "encryptionkey@0"
I20260929 19:10:10.197170 10543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:10.216996 10543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:10.219116 10543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:10.219247 10543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:10.219472 10543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:10.219796 10543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:10.219832 10543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:10.219854 10543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:10.219867 10543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:10.226387 10543 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:42303
I20260929 19:10:10.226511 10671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:42303 every 8 connection(s)
I20260929 19:10:10.226770 10543 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
Sep 29 19:10:10 dist-test-slave-wftq krb5kdc[10167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709010, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:10.235981 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 10543
I20260929 19:10:10.236088 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:10:10.236272 10160 external_mini_cluster.cc:1428] Setting key 82c5e94d431c3cf5007c56302e78b345
I20260929 19:10:10.237027 10255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:10.227557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:41565 (local address 127.9.236.62:43713)
0929 19:10:10.227645 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:10.227647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:10.227798 (+ 151us) server_negotiation.cc:407] Connection header received
0929 19:10:10.228296 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:10.228297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:10.228309 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:10.228380 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:10.229386 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:10.229939 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:10.230990 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:10.231218 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:10.231280 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:10.233866 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:10.233867 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:10.233867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:10.233875 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:10.235378 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:10.235804 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:10.235805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:10.235805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:10.235861 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:10.236192 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:10.236194 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:10.236194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:10.236322 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:10.236397 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:10.236832 (+ 435us) server_negotiation.cc:299] Negotiation successful
0929 19:10:10.236943 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 19:10:10.237370 10674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:10.227498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:43713 (local address 127.9.236.3:41565)
0929 19:10:10.227662 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:10.227704 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:10.228199 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:10.228443 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:10.228446 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:10.228992 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:10.229264 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:10.229269 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:10.230056 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:10.230057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:10.230897 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:10.230899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:10.231292 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:10.231292 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:10.231392 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:10.231988 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:10.231996 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:10.233754 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:10.235468 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:10.235474 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:10.235475 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:10.235691 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:10.235933 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:10.235933 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:10.235934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:10.236074 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:10.236437 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:10.236446 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:10.236647 (+ 201us) client_negotiation.cc:769] Sending connection context
0929 19:10:10.236773 (+ 126us) client_negotiation.cc:240] Negotiation successful
0929 19:10:10.236982 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260929 19:10:10.237970 10672 heartbeater.cc:347] Connected to a master server at 127.9.236.62:43713
I20260929 19:10:10.238078 10672 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:10.238279 10672 heartbeater.cc:511] Master 127.9.236.62:43713 requested a full tablet report, sending...
I20260929 19:10:10.238651 10212 ts_manager.cc:194] Registered new tserver with Master: 678f33cc3d034716adb04c180f5c9072 (127.9.236.3:42303)
I20260929 19:10:10.239086 10212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:41565
I20260929 19:10:10.248749 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 19:10:10 dist-test-slave-wftq krb5kdc[10167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709010, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 19:10:10 dist-test-slave-wftq krb5kdc[10167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709010, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.236.1@KRBTEST.COM
I20260929 19:10:10.273419 10679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:10.265415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47108 (local address 127.9.236.1:37683)
0929 19:10:10.265636 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:10.265643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:10.265657 (+ 14us) server_negotiation.cc:407] Connection header received
0929 19:10:10.265690 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:10.265692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:10.265711 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:10.265812 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:10.266350 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:10.267139 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:10.267861 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:10.268216 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:10.268285 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:10.270373 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:10.270380 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:10.270380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:10.270387 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:10.272094 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:10.272547 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:10.272551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:10.272551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:10.272628 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:10.272835 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:10.272836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:10.272836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:10.272985 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:10.273213 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:10.273255 (+ 42us) server_negotiation.cc:299] Negotiation successful
0929 19:10:10.273326 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":98,"threads_started":1}
W20260929 19:10:10.273793 10318 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47108
Sep 29 19:10:10 dist-test-slave-wftq krb5kdc[10167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 19:10:10 dist-test-slave-wftq krb5kdc[10167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.1@KRBTEST.COM
I20260929 19:10:10.296721 10679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:10.289526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47118 (local address 127.9.236.1:37683)
0929 19:10:10.289600 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:10.289602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:10.289702 (+ 100us) server_negotiation.cc:407] Connection header received
0929 19:10:10.289788 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:10.289789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:10.289801 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:10.289856 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:10.290421 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:10.290983 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:10.291885 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:10.292091 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:10.292154 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:10.294540 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:10.294541 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:10.294542 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:10.294552 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:10.295765 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:10.296062 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:10.296063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:10.296064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:10.296110 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:10.296264 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:10.296265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:10.296265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:10.296355 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:10.296458 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:10.296543 (+ 85us) server_negotiation.cc:299] Negotiation successful
0929 19:10:10.296622 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 19:10:10.297398 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 10275
I20260929 19:10:10.303220 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 10409
I20260929 19:10:10.307886 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 10543
I20260929 19:10:10.311897 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 10180
2026-09-29T19:10:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1798 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:10:10 dist-test-slave-wftq krb5kdc[10686](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:10:10 dist-test-slave-wftq krb5kdc[10686](info): set up 2 sockets
Sep 29 19:10:10 dist-test-slave-wftq krb5kdc[10686](info): commencing operation
krb5kdc: starting...
W20260929 19:10:11.264881 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:10:11 dist-test-slave-wftq krb5kdc[10686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:10:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:10:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:10:11.383097 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:36685
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:41979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:36685
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:11.469494 10699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:11.469730 10699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:11.469761 10699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:11.470954 10699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:11.471005 10699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:11.471025 10699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:11.471041 10699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:11.471064 10699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:11.472576 10699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:41979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:36685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:36685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:11.472826 10699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:11.473049 10699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:11.478019 10704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:11.478055 10705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:11.478055 10707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:11.478394 10699 server_base.cc:1034] running on GCE node
I20260929 19:10:11.478639 10699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:11.478940 10699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:11.480077 10699 hybrid_clock.cc:648] HybridClock initialized: now 1790709011480062 us; error 36 us; skew 500 ppm
Sep 29 19:10:11 dist-test-slave-wftq krb5kdc[10686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709011, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:11.482646 10699 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:10:11.483101 10699 webserver.cc:460] Webserver started at http://127.9.236.62:45367/ using document root <none> and password file <none>
I20260929 19:10:11.483289 10699 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:11.483330 10699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:11.483430 10699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:11.484485 10699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "4719c3cbc34146a1bd62df48a86528e6"
format_stamp: "Formatted at 2026-09-29 19:10:11 on dist-test-slave-wftq"
server_key: "7d90cdf333a51cb1b91537723ff12e4c"
server_key_iv: "a16c2c97a6255c70f542f8a7af4559ae"
server_key_version: "encryptionkey@0"
I20260929 19:10:11.484799 10699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "4719c3cbc34146a1bd62df48a86528e6"
format_stamp: "Formatted at 2026-09-29 19:10:11 on dist-test-slave-wftq"
server_key: "7d90cdf333a51cb1b91537723ff12e4c"
server_key_iv: "a16c2c97a6255c70f542f8a7af4559ae"
server_key_version: "encryptionkey@0"
I20260929 19:10:11.486153 10699 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:10:11.486919 10714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:11.487171 10699 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:11.487269 10699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "4719c3cbc34146a1bd62df48a86528e6"
format_stamp: "Formatted at 2026-09-29 19:10:11 on dist-test-slave-wftq"
server_key: "7d90cdf333a51cb1b91537723ff12e4c"
server_key_iv: "a16c2c97a6255c70f542f8a7af4559ae"
server_key_version: "encryptionkey@0"
I20260929 19:10:11.487357 10699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:11.517379 10699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:11.519618 10699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:11.519747 10699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:11.524407 10699 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:36685
I20260929 19:10:11.524832 10699 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:10:11.524791 10766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:36685 every 8 connection(s)
I20260929 19:10:11.525619 10767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:11.528074 10767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6: Bootstrap starting.
I20260929 19:10:11.528193 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 10699
I20260929 19:10:11.528275 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:10:11.528429 10160 external_mini_cluster.cc:1428] Setting key 57bae7d9198f369b933f1d5815db0466
I20260929 19:10:11.528776 10767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:11.529083 10767 log.cc:824] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:11.529759 10767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6: No bootstrap required, opened a new log
I20260929 19:10:11.531706 10767 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4719c3cbc34146a1bd62df48a86528e6" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36685 } }
I20260929 19:10:11.531844 10767 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:11.531875 10767 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4719c3cbc34146a1bd62df48a86528e6, State: Initialized, Role: FOLLOWER
I20260929 19:10:11.531996 10767 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4719c3cbc34146a1bd62df48a86528e6" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36685 } }
I20260929 19:10:11.532073 10767 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:10:11.532099 10767 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:10:11.532143 10767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:11.532711 10767 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4719c3cbc34146a1bd62df48a86528e6" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36685 } }
I20260929 19:10:11.532809 10767 leader_election.cc:302] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4719c3cbc34146a1bd62df48a86528e6; no voters:
I20260929 19:10:11.532959 10767 leader_election.cc:288] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:10:11.533001 10772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:11.533151 10772 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6 [term 1 LEADER]: Becoming Leader. State: Replica: 4719c3cbc34146a1bd62df48a86528e6, State: Running, Role: LEADER
I20260929 19:10:11.533258 10767 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:11.533252 10772 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4719c3cbc34146a1bd62df48a86528e6" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36685 } }
I20260929 19:10:11.533618 10772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4719c3cbc34146a1bd62df48a86528e6. Latest consensus state: current_term: 1 leader_uuid: "4719c3cbc34146a1bd62df48a86528e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4719c3cbc34146a1bd62df48a86528e6" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36685 } } }
I20260929 19:10:11.533694 10772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:11.533891 10777 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:11.533841 10773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4719c3cbc34146a1bd62df48a86528e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4719c3cbc34146a1bd62df48a86528e6" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36685 } } }
I20260929 19:10:11.533980 10773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:11.534432 10777 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 19:10:11 dist-test-slave-wftq krb5kdc[10686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:11.536042 10777 catalog_manager.cc:1347] Generated new cluster ID: 5fa8d12a4e354768a44e1a7b9f89de78
I20260929 19:10:11.536094 10777 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:10:11.540449 10770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:11.528955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43762 (local address 127.9.236.62:36685)
0929 19:10:11.529188 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:11.529200 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:11.529241 (+ 41us) server_negotiation.cc:407] Connection header received
0929 19:10:11.529812 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:11.529819 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:11.529884 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:11.530050 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:11.530684 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:11.531663 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:11.532550 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:11.533119 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:11.533538 (+ 419us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:11.536105 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:11.536110 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:11.536111 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:11.536124 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:11.538252 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:11.538674 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:11.538677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:11.538678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:11.538788 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:11.539004 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:11.539005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:11.539005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:11.539322 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:11.539966 (+ 644us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:11.540053 (+ 87us) server_negotiation.cc:299] Negotiation successful
0929 19:10:11.540208 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":99,"threads_started":1}
I20260929 19:10:11.546816 10777 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:10:11.547335 10777 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:11.553543 10777 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4719c3cbc34146a1bd62df48a86528e6: Generated new TSK 0
I20260929 19:10:11.553712 10777 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.236.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
I20260929 19:10:11.588627 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:36685
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:41979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:11.698400 10793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:11.698634 10793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:11.698657 10793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:11.699815 10793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:11.699851 10793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:11.699877 10793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:10:11.701421 10793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:41979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:36685
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:11.701628 10793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:11.701835 10793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:11.707194 10799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:11.707309 10793 server_base.cc:1034] running on GCE node
W20260929 19:10:11.707170 10798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:11.707170 10801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:11.707680 10793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:11.708022 10793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:11.709172 10793 hybrid_clock.cc:648] HybridClock initialized: now 1790709011709146 us; error 35 us; skew 500 ppm
Sep 29 19:10:11 dist-test-slave-wftq krb5kdc[10686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709011, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:11.711903 10793 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:10:11.712368 10793 webserver.cc:460] Webserver started at http://127.9.236.1:39409/ using document root <none> and password file <none>
I20260929 19:10:11.712597 10793 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:11.712647 10793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:11.712775 10793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:11.713940 10793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "d02b6a3f340049b7be148635896327a3"
format_stamp: "Formatted at 2026-09-29 19:10:11 on dist-test-slave-wftq"
server_key: "b3f129ceeb1c9a2c094572e9cf45fc12"
server_key_iv: "17724c9f8774fe84523509daa01a8d83"
server_key_version: "encryptionkey@0"
I20260929 19:10:11.714277 10793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "d02b6a3f340049b7be148635896327a3"
format_stamp: "Formatted at 2026-09-29 19:10:11 on dist-test-slave-wftq"
server_key: "b3f129ceeb1c9a2c094572e9cf45fc12"
server_key_iv: "17724c9f8774fe84523509daa01a8d83"
server_key_version: "encryptionkey@0"
I20260929 19:10:11.715567 10793 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 19:10:11.716377 10808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:11.716574 10793 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:11.716662 10793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "d02b6a3f340049b7be148635896327a3"
format_stamp: "Formatted at 2026-09-29 19:10:11 on dist-test-slave-wftq"
server_key: "b3f129ceeb1c9a2c094572e9cf45fc12"
server_key_iv: "17724c9f8774fe84523509daa01a8d83"
server_key_version: "encryptionkey@0"
I20260929 19:10:11.716722 10793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:11.761948 10793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:11.764387 10793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:11.764539 10793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:11.764796 10793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:11.765240 10793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:11.765280 10793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:11.765304 10793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:11.765316 10793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:11.772167 10793 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:35107
I20260929 19:10:11.772234 10921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:35107 every 8 connection(s)
I20260929 19:10:11.772567 10793 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:10:11.775061 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 10793
I20260929 19:10:11.775141 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:10:11.775300 10160 external_mini_cluster.cc:1428] Setting key 99db03e4c136b006236f58c3e56fd638
Sep 29 19:10:11 dist-test-slave-wftq krb5kdc[10686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709011, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:11.782250 10770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:11.773399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:41607 (local address 127.9.236.62:36685)
0929 19:10:11.773474 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:11.773476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:11.773693 (+ 217us) server_negotiation.cc:407] Connection header received
0929 19:10:11.774661 (+ 968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:11.774662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:11.774674 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:11.774725 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:11.775418 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:11.776093 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:11.777077 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:11.777398 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:11.777463 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:11.779679 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:11.779681 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:11.779681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:11.779689 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:11.780965 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:11.781378 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:11.781379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:11.781379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:11.781427 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:11.781642 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:11.781643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:11.781644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:11.781750 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:11.781807 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:11.782071 (+ 264us) server_negotiation.cc:299] Negotiation successful
0929 19:10:11.782156 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 19:10:11.782436 10924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:11.773358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:36685 (local address 127.9.236.1:41607)
0929 19:10:11.773567 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:11.773605 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:11.774554 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:11.774804 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:11.774808 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:11.775079 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:11.775305 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:11.775310 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:11.776183 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:11.776183 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:11.776994 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:11.776997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:11.777509 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:11.777511 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:11.777638 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:11.778128 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:11.778136 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:11.779592 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:11.781051 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:11.781057 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:11.781060 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:11.781315 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:11.781484 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:11.781485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:11.781486 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:11.781590 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:11.781803 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:11.781809 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:11.781950 (+ 141us) client_negotiation.cc:769] Sending connection context
0929 19:10:11.782045 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 19:10:11.782213 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":93,"threads_started":1}
I20260929 19:10:11.782887 10922 heartbeater.cc:347] Connected to a master server at 127.9.236.62:36685
I20260929 19:10:11.782981 10922 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.9.236.2@KRBTEST.COM; defaulting to no policy
I20260929 19:10:11.783157 10922 heartbeater.cc:511] Master 127.9.236.62:36685 requested a full tablet report, sending...
I20260929 19:10:11.783727 10731 ts_manager.cc:194] Registered new tserver with Master: d02b6a3f340049b7be148635896327a3 (127.9.236.1:35107)
I20260929 19:10:11.784382 10731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:41607
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
I20260929 19:10:11.799840 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:36685
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:41979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:11.882261 10927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:11.882465 10927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:11.882488 10927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:11.883576 10927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:11.883633 10927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:11.883659 10927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:10:11.885162 10927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:41979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:36685
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:11.885376 10927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:11.885592 10927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:11.890645 10935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:11.890655 10932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:11.890728 10933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:11.890955 10927 server_base.cc:1034] running on GCE node
I20260929 19:10:11.891202 10927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:11.891443 10927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:11.892603 10927 hybrid_clock.cc:648] HybridClock initialized: now 1790709011892573 us; error 56 us; skew 500 ppm
Sep 29 19:10:11 dist-test-slave-wftq krb5kdc[10686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709011, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:11.895260 10927 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:10:11.895757 10927 webserver.cc:460] Webserver started at http://127.9.236.2:41079/ using document root <none> and password file <none>
I20260929 19:10:11.895961 10927 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:11.896010 10927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:11.896131 10927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:11.897323 10927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "b6242c4b190e408782b3cc48a95bef03"
format_stamp: "Formatted at 2026-09-29 19:10:11 on dist-test-slave-wftq"
server_key: "f49318f5a8a0e4263025f892f5c53ec8"
server_key_iv: "e0eb36da39ca30bc3390c38b7851eb5b"
server_key_version: "encryptionkey@0"
I20260929 19:10:11.897650 10927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "b6242c4b190e408782b3cc48a95bef03"
format_stamp: "Formatted at 2026-09-29 19:10:11 on dist-test-slave-wftq"
server_key: "f49318f5a8a0e4263025f892f5c53ec8"
server_key_iv: "e0eb36da39ca30bc3390c38b7851eb5b"
server_key_version: "encryptionkey@0"
I20260929 19:10:11.898962 10927 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:10:11.899670 10942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:11.899847 10927 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:10:11.899930 10927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "b6242c4b190e408782b3cc48a95bef03"
format_stamp: "Formatted at 2026-09-29 19:10:11 on dist-test-slave-wftq"
server_key: "f49318f5a8a0e4263025f892f5c53ec8"
server_key_iv: "e0eb36da39ca30bc3390c38b7851eb5b"
server_key_version: "encryptionkey@0"
I20260929 19:10:11.899988 10927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:11.917289 10927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:11.919365 10927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:11.919498 10927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:11.919723 10927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:11.920011 10927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:11.920045 10927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:11.920073 10927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:11.920090 10927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:11.926434 10927 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:44611
I20260929 19:10:11.926499 11055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:44611 every 8 connection(s)
I20260929 19:10:11.926801 10927 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
Sep 29 19:10:11 dist-test-slave-wftq krb5kdc[10686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709011, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:11.935098 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 10927
I20260929 19:10:11.935187 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:10:11.935384 10160 external_mini_cluster.cc:1428] Setting key deb932df828ace0c1a0fd2b8dfef14e2
I20260929 19:10:11.936245 10770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:11.927636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:51757 (local address 127.9.236.62:36685)
0929 19:10:11.927728 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:11.927730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:11.927940 (+ 210us) server_negotiation.cc:407] Connection header received
0929 19:10:11.928873 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:11.928875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:11.928887 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:11.928940 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:11.929595 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:11.930095 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:11.931047 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:11.931229 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:11.931283 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:11.933536 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:11.933538 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:11.933539 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:11.933547 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:11.934813 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:11.935179 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:11.935180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:11.935181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:11.935254 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:11.935530 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:11.935531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:11.935532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:11.935685 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:11.935766 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:11.936067 (+ 301us) server_negotiation.cc:299] Negotiation successful
0929 19:10:11.936153 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260929 19:10:11.936371 11058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:11.927603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:36685 (local address 127.9.236.2:51757)
0929 19:10:11.927817 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:11.927854 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:11.928768 (+ 914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:11.929015 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:11.929019 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:11.929319 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:11.929508 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:11.929513 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:11.930171 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:11.930172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:11.930968 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:11.930970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:11.931294 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:11.931295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:11.931418 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:11.931886 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:11.931893 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:11.933438 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:11.934894 (+ 1456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:11.934899 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:11.934901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:11.935104 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:11.935317 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:11.935318 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:11.935318 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:11.935419 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:11.935764 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:11.935770 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:11.935919 (+ 149us) client_negotiation.cc:769] Sending connection context
0929 19:10:11.936032 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 19:10:11.936190 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":113,"threads_started":1}
I20260929 19:10:11.936859 11056 heartbeater.cc:347] Connected to a master server at 127.9.236.62:36685
I20260929 19:10:11.936982 11056 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:11.937232 11056 heartbeater.cc:511] Master 127.9.236.62:36685 requested a full tablet report, sending...
I20260929 19:10:11.937676 10731 ts_manager.cc:194] Registered new tserver with Master: b6242c4b190e408782b3cc48a95bef03 (127.9.236.2:44611)
I20260929 19:10:11.938093 10731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:51757
WARNING: no policy specified for kudu/127.9.236.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
I20260929 19:10:11.960039 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:36685
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:41979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:12.047883 11061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:12.048137 11061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:12.048162 11061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:12.049397 11061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:12.049438 11061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:12.049465 11061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:10:12.051101 11061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:41979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:36685
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:12.051334 11061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:12.051553 11061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:12.057003 11067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:12.057093 11066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:12.056984 11069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:12.057219 11061 server_base.cc:1034] running on GCE node
I20260929 19:10:12.057516 11061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:12.057750 11061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:12.058902 11061 hybrid_clock.cc:648] HybridClock initialized: now 1790709012058874 us; error 50 us; skew 500 ppm
Sep 29 19:10:12 dist-test-slave-wftq krb5kdc[10686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709012, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:12.061811 11061 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:10:12.062338 11061 webserver.cc:460] Webserver started at http://127.9.236.3:37183/ using document root <none> and password file <none>
I20260929 19:10:12.062556 11061 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:12.062610 11061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:12.062752 11061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:12.064008 11061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "08b6b0e10f2d4ba0b550505e95eee7a8"
format_stamp: "Formatted at 2026-09-29 19:10:12 on dist-test-slave-wftq"
server_key: "91a518f907611804fc68d6f379dcf4f3"
server_key_iv: "d09a654f4b6d0b507ec8fc05d4baae52"
server_key_version: "encryptionkey@0"
I20260929 19:10:12.064378 11061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "08b6b0e10f2d4ba0b550505e95eee7a8"
format_stamp: "Formatted at 2026-09-29 19:10:12 on dist-test-slave-wftq"
server_key: "91a518f907611804fc68d6f379dcf4f3"
server_key_iv: "d09a654f4b6d0b507ec8fc05d4baae52"
server_key_version: "encryptionkey@0"
I20260929 19:10:12.065892 11061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:10:12.066787 11076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:12.067008 11061 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:12.067095 11061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "08b6b0e10f2d4ba0b550505e95eee7a8"
format_stamp: "Formatted at 2026-09-29 19:10:12 on dist-test-slave-wftq"
server_key: "91a518f907611804fc68d6f379dcf4f3"
server_key_iv: "d09a654f4b6d0b507ec8fc05d4baae52"
server_key_version: "encryptionkey@0"
I20260929 19:10:12.067157 11061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:12.083060 11061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:12.085292 11061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:12.085434 11061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:12.085687 11061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:12.086002 11061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:12.086037 11061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:12.086069 11061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:12.086093 11061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:12.093086 11061 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:43147
I20260929 19:10:12.093205 11189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:43147 every 8 connection(s)
I20260929 19:10:12.093508 11061 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:10:12.095363 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 11061
I20260929 19:10:12.095469 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:10:12.095625 10160 external_mini_cluster.cc:1428] Setting key bb8f32d32d4b322ed642fcd953f6ded9
Sep 29 19:10:12 dist-test-slave-wftq krb5kdc[10686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709012, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:12.102864 10770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:12.094298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:59833 (local address 127.9.236.62:36685)
0929 19:10:12.094356 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:12.094358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:12.094579 (+ 221us) server_negotiation.cc:407] Connection header received
0929 19:10:12.095658 (+ 1079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:12.095659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:12.095673 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:12.095749 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:12.096465 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.096970 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.097987 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.098183 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.098237 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:12.100314 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:12.100315 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:12.100316 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:12.100322 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:12.101626 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:12.101981 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:12.101982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:12.101983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:12.102033 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:12.102233 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:12.102233 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:12.102234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:12.102342 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:12.102399 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:12.102693 (+ 294us) server_negotiation.cc:299] Negotiation successful
0929 19:10:12.102777 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 19:10:12.103006 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:12.094261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:36685 (local address 127.9.236.3:59833)
0929 19:10:12.094450 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:12.094490 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:12.095545 (+ 1055us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:12.095827 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:12.095830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:12.096121 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:12.096352 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:12.096357 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.097049 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.097050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:12.097908 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:12.097910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.098255 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.098256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:12.098364 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:12.098833 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:12.098841 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:12.100215 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:12.101697 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:12.101703 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:12.101706 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:12.101914 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:12.102084 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:12.102085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:12.102085 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:12.102180 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:12.102396 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:12.102403 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:12.102531 (+ 128us) client_negotiation.cc:769] Sending connection context
0929 19:10:12.102653 (+ 122us) client_negotiation.cc:240] Negotiation successful
0929 19:10:12.102798 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":107,"threads_started":1}
I20260929 19:10:12.103482 11190 heartbeater.cc:347] Connected to a master server at 127.9.236.62:36685
I20260929 19:10:12.103569 11190 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:12.103744 11190 heartbeater.cc:511] Master 127.9.236.62:36685 requested a full tablet report, sending...
I20260929 19:10:12.104112 10731 ts_manager.cc:194] Registered new tserver with Master: 08b6b0e10f2d4ba0b550505e95eee7a8 (127.9.236.3:43147)
I20260929 19:10:12.104473 10731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:59833
I20260929 19:10:12.107228 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:10:12.114792 10770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:12.108925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43770 (local address 127.9.236.62:36685)
0929 19:10:12.109015 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:12.109017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:12.109131 (+ 114us) server_negotiation.cc:407] Connection header received
0929 19:10:12.109227 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:12.109228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:12.109238 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:12.109285 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:12.109850 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.110396 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.111109 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.111296 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.111349 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:12.112480 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:12.112482 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:12.112483 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:12.112489 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:12.113773 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:12.114082 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:12.114083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:12.114084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:12.114132 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:12.114282 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:12.114283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:12.114283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:12.114426 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:12.114522 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:12.114623 (+ 101us) server_negotiation.cc:299] Negotiation successful
0929 19:10:12.114695 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 19:10:12.116950 10731 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43770:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:10:12.118608 10731 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:10:12.125694 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:12.121557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41308 (local address 127.9.236.1:35107)
0929 19:10:12.121950 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:12.121955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:12.121974 (+ 19us) server_negotiation.cc:407] Connection header received
0929 19:10:12.122016 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:12.122018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:12.122035 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:12.122114 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:12.122317 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.123477 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.124721 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.125342 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.125415 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:12.125483 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:12.125575 (+ 92us) server_negotiation.cc:299] Negotiation successful
0929 19:10:12.125614 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":97,"threads_started":1}
I20260929 19:10:12.125705 11201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:12.121411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:35107 (local address 127.0.0.1:41308)
0929 19:10:12.121741 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:12.121753 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:12.121814 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:12.122197 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:12.122198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:12.122201 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:12.122245 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:12.122246 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.123596 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.123597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:12.124640 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:12.124641 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.125421 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.125423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:12.125490 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:12.125496 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 19:10:12.125554 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 19:10:12.125606 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":94,"threads_started":1}
I20260929 19:10:12.125824 11206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:12.121819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38572 (local address 127.9.236.3:43147)
0929 19:10:12.122017 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:12.122020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:12.122030 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:10:12.122063 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:12.122065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:12.122079 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:12.122152 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:12.122327 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.123109 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.124502 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.125215 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.125299 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:12.125354 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:12.125668 (+ 314us) server_negotiation.cc:299] Negotiation successful
0929 19:10:12.125737 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":99,"threads_started":1}
I20260929 19:10:12.125968 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:12.121543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:43147 (local address 127.0.0.1:38572)
0929 19:10:12.121794 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:12.121803 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:12.121845 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:12.122170 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:12.122171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:12.122174 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:12.122239 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:12.122242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.123202 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.123203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:12.124410 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:12.124413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.125304 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.125306 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:12.125396 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:12.125414 (+ 18us) client_negotiation.cc:769] Sending connection context
0929 19:10:12.125500 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 19:10:12.125580 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":1,"thread_start_us":104,"threads_started":1}
I20260929 19:10:12.126534 10856 tablet_service.cc:1467] Processing CreateTablet for tablet 2759273b58a444d9997e6052164bf1c8 (DEFAULT_TABLE table=test-table [id=b9675555989c41a6a91107b1a3300121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:12.126754 11124 tablet_service.cc:1467] Processing CreateTablet for tablet 2759273b58a444d9997e6052164bf1c8 (DEFAULT_TABLE table=test-table [id=b9675555989c41a6a91107b1a3300121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:12.126942 10856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2759273b58a444d9997e6052164bf1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:12.127110 11124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2759273b58a444d9997e6052164bf1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:12.127624 11202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:12.121353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:44611 (local address 127.0.0.1:40790)
0929 19:10:12.121592 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:12.121639 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:12.121706 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:12.121942 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:12.121946 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:12.121948 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:12.122029 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:12.122031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.124739 (+ 2708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.124740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:12.125741 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:12.125743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.127396 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.127397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:12.127459 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:12.127464 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:10:12.127510 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 19:10:12.127551 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":46,"threads_started":1}
I20260929 19:10:12.128028 11203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:12.121490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40790 (local address 127.9.236.2:44611)
0929 19:10:12.121675 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:12.121680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:12.121692 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:10:12.121834 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:12.121836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:12.121852 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:12.121931 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:12.122323 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.123458 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.126145 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.127237 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.127303 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:12.127361 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:12.127910 (+ 549us) server_negotiation.cc:299] Negotiation successful
0929 19:10:12.127957 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":92,"threads_started":1}
I20260929 19:10:12.129009 10990 tablet_service.cc:1467] Processing CreateTablet for tablet 2759273b58a444d9997e6052164bf1c8 (DEFAULT_TABLE table=test-table [id=b9675555989c41a6a91107b1a3300121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:12.129352 10990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2759273b58a444d9997e6052164bf1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:12.130208 11207 tablet_bootstrap.cc:492] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8: Bootstrap starting.
I20260929 19:10:12.130587 11208 tablet_bootstrap.cc:492] T 2759273b58a444d9997e6052164bf1c8 P d02b6a3f340049b7be148635896327a3: Bootstrap starting.
I20260929 19:10:12.130851 11207 tablet_bootstrap.cc:654] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:12.131193 11208 tablet_bootstrap.cc:654] T 2759273b58a444d9997e6052164bf1c8 P d02b6a3f340049b7be148635896327a3: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:12.131322 11207 log.cc:824] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:12.131713 11208 log.cc:824] T 2759273b58a444d9997e6052164bf1c8 P d02b6a3f340049b7be148635896327a3: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:12.131985 11207 tablet_bootstrap.cc:492] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8: No bootstrap required, opened a new log
I20260929 19:10:12.132048 11207 ts_tablet_manager.cc:1397] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:10:12.132531 11208 tablet_bootstrap.cc:492] T 2759273b58a444d9997e6052164bf1c8 P d02b6a3f340049b7be148635896327a3: No bootstrap required, opened a new log
I20260929 19:10:12.132591 11208 ts_tablet_manager.cc:1397] T 2759273b58a444d9997e6052164bf1c8 P d02b6a3f340049b7be148635896327a3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:10:12.133838 11207 raft_consensus.cc:348] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6242c4b190e408782b3cc48a95bef03" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 44611 } } peers { permanent_uuid: "08b6b0e10f2d4ba0b550505e95eee7a8" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43147 } } peers { permanent_uuid: "d02b6a3f340049b7be148635896327a3" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 35107 } }
I20260929 19:10:12.134011 11207 raft_consensus.cc:374] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:12.134033 11207 raft_consensus.cc:729] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08b6b0e10f2d4ba0b550505e95eee7a8, State: Initialized, Role: FOLLOWER
I20260929 19:10:12.134143 11207 consensus_queue.cc:260] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6242c4b190e408782b3cc48a95bef03" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 44611 } } peers { permanent_uuid: "08b6b0e10f2d4ba0b550505e95eee7a8" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43147 } } peers { permanent_uuid: "d02b6a3f340049b7be148635896327a3" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 35107 } }
I20260929 19:10:12.134140 11208 raft_consensus.cc:348] T 2759273b58a444d9997e6052164bf1c8 P d02b6a3f340049b7be148635896327a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6242c4b190e408782b3cc48a95bef03" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 44611 } } peers { permanent_uuid: "08b6b0e10f2d4ba0b550505e95eee7a8" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43147 } } peers { permanent_uuid: "d02b6a3f340049b7be148635896327a3" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 35107 } }
I20260929 19:10:12.134354 11208 raft_consensus.cc:374] T 2759273b58a444d9997e6052164bf1c8 P d02b6a3f340049b7be148635896327a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:12.134389 11208 raft_consensus.cc:729] T 2759273b58a444d9997e6052164bf1c8 P d02b6a3f340049b7be148635896327a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02b6a3f340049b7be148635896327a3, State: Initialized, Role: FOLLOWER
I20260929 19:10:12.134424 11207 ts_tablet_manager.cc:1428] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 19:10:12.134470 11190 heartbeater.cc:503] Master 127.9.236.62:36685 was elected leader, sending a full tablet report...
I20260929 19:10:12.134507 11208 consensus_queue.cc:260] T 2759273b58a444d9997e6052164bf1c8 P d02b6a3f340049b7be148635896327a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6242c4b190e408782b3cc48a95bef03" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 44611 } } peers { permanent_uuid: "08b6b0e10f2d4ba0b550505e95eee7a8" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43147 } } peers { permanent_uuid: "d02b6a3f340049b7be148635896327a3" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 35107 } }
I20260929 19:10:12.134742 11208 ts_tablet_manager.cc:1428] T 2759273b58a444d9997e6052164bf1c8 P d02b6a3f340049b7be148635896327a3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:10:12.134796 10922 heartbeater.cc:503] Master 127.9.236.62:36685 was elected leader, sending a full tablet report...
I20260929 19:10:12.134948 11211 tablet_bootstrap.cc:492] T 2759273b58a444d9997e6052164bf1c8 P b6242c4b190e408782b3cc48a95bef03: Bootstrap starting.
I20260929 19:10:12.135552 11211 tablet_bootstrap.cc:654] T 2759273b58a444d9997e6052164bf1c8 P b6242c4b190e408782b3cc48a95bef03: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:12.136018 11211 log.cc:824] T 2759273b58a444d9997e6052164bf1c8 P b6242c4b190e408782b3cc48a95bef03: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:12.136649 11211 tablet_bootstrap.cc:492] T 2759273b58a444d9997e6052164bf1c8 P b6242c4b190e408782b3cc48a95bef03: No bootstrap required, opened a new log
I20260929 19:10:12.136706 11211 ts_tablet_manager.cc:1397] T 2759273b58a444d9997e6052164bf1c8 P b6242c4b190e408782b3cc48a95bef03: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:10:12.138137 11211 raft_consensus.cc:348] T 2759273b58a444d9997e6052164bf1c8 P b6242c4b190e408782b3cc48a95bef03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6242c4b190e408782b3cc48a95bef03" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 44611 } } peers { permanent_uuid: "08b6b0e10f2d4ba0b550505e95eee7a8" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43147 } } peers { permanent_uuid: "d02b6a3f340049b7be148635896327a3" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 35107 } }
I20260929 19:10:12.138283 11211 raft_consensus.cc:374] T 2759273b58a444d9997e6052164bf1c8 P b6242c4b190e408782b3cc48a95bef03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:12.138324 11211 raft_consensus.cc:729] T 2759273b58a444d9997e6052164bf1c8 P b6242c4b190e408782b3cc48a95bef03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6242c4b190e408782b3cc48a95bef03, State: Initialized, Role: FOLLOWER
I20260929 19:10:12.138434 11211 consensus_queue.cc:260] T 2759273b58a444d9997e6052164bf1c8 P b6242c4b190e408782b3cc48a95bef03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6242c4b190e408782b3cc48a95bef03" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 44611 } } peers { permanent_uuid: "08b6b0e10f2d4ba0b550505e95eee7a8" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43147 } } peers { permanent_uuid: "d02b6a3f340049b7be148635896327a3" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 35107 } }
I20260929 19:10:12.138659 11211 ts_tablet_manager.cc:1428] T 2759273b58a444d9997e6052164bf1c8 P b6242c4b190e408782b3cc48a95bef03: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:10:12.138722 11056 heartbeater.cc:503] Master 127.9.236.62:36685 was elected leader, sending a full tablet report...
W20260929 19:10:12.177399 11057 tablet.cc:2369] T 2759273b58a444d9997e6052164bf1c8 P b6242c4b190e408782b3cc48a95bef03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:10:12.223665 11212 raft_consensus.cc:482] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:10:12.223869 11212 raft_consensus.cc:504] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6242c4b190e408782b3cc48a95bef03" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 44611 } } peers { permanent_uuid: "08b6b0e10f2d4ba0b550505e95eee7a8" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43147 } } peers { permanent_uuid: "d02b6a3f340049b7be148635896327a3" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 35107 } }
I20260929 19:10:12.224325 11212 leader_election.cc:288] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6242c4b190e408782b3cc48a95bef03 (127.9.236.2:44611), d02b6a3f340049b7be148635896327a3 (127.9.236.1:35107)
I20260929 19:10:12.227505 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:12.224542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:44611 (local address 127.9.236.3:42995)
0929 19:10:12.224596 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:12.224608 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:12.224661 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:12.224787 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:12.224787 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:12.224789 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:12.224873 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:12.224875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.225748 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.225750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:12.226573 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:12.226576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.227255 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.227256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:12.227336 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:12.227341 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:10:12.227398 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 19:10:12.227437 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260929 19:10:12.227552 11203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:12.224643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:42995 (local address 127.9.236.2:44611)
0929 19:10:12.224685 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:12.224687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:12.224696 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:12.224725 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:12.224726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:12.224736 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:12.224783 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:12.225025 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.225647 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.226694 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.227168 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.227227 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:12.227238 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:12.227421 (+ 183us) server_negotiation.cc:299] Negotiation successful
0929 19:10:12.227468 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:10:12.227638 11216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:12.224561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:35107 (local address 127.9.236.3:50061)
0929 19:10:12.224796 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:12.224806 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:12.224879 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:12.225029 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:12.225030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:12.225032 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:12.225147 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:12.225149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.225862 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.225863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:12.226765 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:12.226774 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.227364 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.227365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:12.227442 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:12.227447 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:10:12.227517 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 19:10:12.227561 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":12,"thread_start_us":149,"threads_started":1}
I20260929 19:10:12.227662 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:12.224654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:50061 (local address 127.9.236.1:35107)
0929 19:10:12.224716 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:12.224718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:12.224896 (+ 178us) server_negotiation.cc:407] Connection header received
0929 19:10:12.224934 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:12.224936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:12.224954 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:12.225009 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:12.225269 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.225761 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.226886 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.227282 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.227352 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:12.227364 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:12.227538 (+ 174us) server_negotiation.cc:299] Negotiation successful
0929 19:10:12.227588 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 19:10:12.227962 10876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2759273b58a444d9997e6052164bf1c8" candidate_uuid: "08b6b0e10f2d4ba0b550505e95eee7a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02b6a3f340049b7be148635896327a3" is_pre_election: true
I20260929 19:10:12.228132 10876 raft_consensus.cc:2454] T 2759273b58a444d9997e6052164bf1c8 P d02b6a3f340049b7be148635896327a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08b6b0e10f2d4ba0b550505e95eee7a8 in term 0.
I20260929 19:10:12.228144 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2759273b58a444d9997e6052164bf1c8" candidate_uuid: "08b6b0e10f2d4ba0b550505e95eee7a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6242c4b190e408782b3cc48a95bef03" is_pre_election: true
I20260929 19:10:12.228327 11010 raft_consensus.cc:2454] T 2759273b58a444d9997e6052164bf1c8 P b6242c4b190e408782b3cc48a95bef03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08b6b0e10f2d4ba0b550505e95eee7a8 in term 0.
I20260929 19:10:12.228364 11079 leader_election.cc:302] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08b6b0e10f2d4ba0b550505e95eee7a8, d02b6a3f340049b7be148635896327a3; no voters:
I20260929 19:10:12.228492 11212 raft_consensus.cc:2781] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:10:12.228549 11212 raft_consensus.cc:482] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:10:12.228578 11212 raft_consensus.cc:3037] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:12.229415 11212 raft_consensus.cc:504] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6242c4b190e408782b3cc48a95bef03" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 44611 } } peers { permanent_uuid: "08b6b0e10f2d4ba0b550505e95eee7a8" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43147 } } peers { permanent_uuid: "d02b6a3f340049b7be148635896327a3" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 35107 } }
I20260929 19:10:12.229573 11212 leader_election.cc:288] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8 [CANDIDATE]: Term 1 election: Requested vote from peers b6242c4b190e408782b3cc48a95bef03 (127.9.236.2:44611), d02b6a3f340049b7be148635896327a3 (127.9.236.1:35107)
I20260929 19:10:12.229735 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2759273b58a444d9997e6052164bf1c8" candidate_uuid: "08b6b0e10f2d4ba0b550505e95eee7a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6242c4b190e408782b3cc48a95bef03"
I20260929 19:10:12.229776 10876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2759273b58a444d9997e6052164bf1c8" candidate_uuid: "08b6b0e10f2d4ba0b550505e95eee7a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02b6a3f340049b7be148635896327a3"
I20260929 19:10:12.229818 11010 raft_consensus.cc:3037] T 2759273b58a444d9997e6052164bf1c8 P b6242c4b190e408782b3cc48a95bef03 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:12.229869 10876 raft_consensus.cc:3037] T 2759273b58a444d9997e6052164bf1c8 P d02b6a3f340049b7be148635896327a3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:12.230765 11010 raft_consensus.cc:2454] T 2759273b58a444d9997e6052164bf1c8 P b6242c4b190e408782b3cc48a95bef03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08b6b0e10f2d4ba0b550505e95eee7a8 in term 1.
I20260929 19:10:12.230782 10876 raft_consensus.cc:2454] T 2759273b58a444d9997e6052164bf1c8 P d02b6a3f340049b7be148635896327a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08b6b0e10f2d4ba0b550505e95eee7a8 in term 1.
I20260929 19:10:12.230964 11079 leader_election.cc:302] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08b6b0e10f2d4ba0b550505e95eee7a8, d02b6a3f340049b7be148635896327a3; no voters:
I20260929 19:10:12.231105 11212 raft_consensus.cc:2781] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:12.231308 11212 raft_consensus.cc:686] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8 [term 1 LEADER]: Becoming Leader. State: Replica: 08b6b0e10f2d4ba0b550505e95eee7a8, State: Running, Role: LEADER
I20260929 19:10:12.231415 11212 consensus_queue.cc:237] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6242c4b190e408782b3cc48a95bef03" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 44611 } } peers { permanent_uuid: "08b6b0e10f2d4ba0b550505e95eee7a8" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43147 } } peers { permanent_uuid: "d02b6a3f340049b7be148635896327a3" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 35107 } }
I20260929 19:10:12.232317 10731 catalog_manager.cc:5495] T 2759273b58a444d9997e6052164bf1c8 P 08b6b0e10f2d4ba0b550505e95eee7a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08b6b0e10f2d4ba0b550505e95eee7a8 (127.9.236.3). New cstate: current_term: 1 leader_uuid: "08b6b0e10f2d4ba0b550505e95eee7a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6242c4b190e408782b3cc48a95bef03" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 44611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08b6b0e10f2d4ba0b550505e95eee7a8" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d02b6a3f340049b7be148635896327a3" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 35107 } health_report { overall_health: UNKNOWN } } }
Sep 29 19:10:12 dist-test-slave-wftq krb5kdc[10686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.1@KRBTEST.COM
I20260929 19:10:12.244660 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:12.237711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41318 (local address 127.9.236.1:35107)
0929 19:10:12.237798 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:12.237800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:12.237903 (+ 103us) server_negotiation.cc:407] Connection header received
0929 19:10:12.237970 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:12.237971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:12.237981 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:12.238030 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:12.238596 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.239071 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.239750 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.239930 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.239981 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:12.242213 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:12.242222 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:12.242223 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:12.242229 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:12.243678 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:12.243986 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:12.243990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:12.243990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:12.244040 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:12.244197 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:12.244198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:12.244199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:12.244323 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:12.244458 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:12.244497 (+ 39us) server_negotiation.cc:299] Negotiation successful
0929 19:10:12.244570 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Sep 29 19:10:12 dist-test-slave-wftq krb5kdc[10686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709012, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 19:10:12 dist-test-slave-wftq krb5kdc[10686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709012, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.236.1@KRBTEST.COM
I20260929 19:10:12.268620 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:12.261804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41330 (local address 127.9.236.1:35107)
0929 19:10:12.261889 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:12.261891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:12.261979 (+ 88us) server_negotiation.cc:407] Connection header received
0929 19:10:12.262057 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:12.262058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:12.262068 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:12.262110 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:12.262650 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.263141 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.263847 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.264053 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.264108 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:12.266474 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:12.266475 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:12.266475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:12.266482 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:12.267712 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:12.267990 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:12.267990 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:12.267991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:12.268037 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:12.268198 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:12.268199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:12.268199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:12.268289 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:12.268356 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:12.268500 (+ 144us) server_negotiation.cc:299] Negotiation successful
0929 19:10:12.268554 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260929 19:10:12.273365 10923 tablet.cc:2369] T 2759273b58a444d9997e6052164bf1c8 P d02b6a3f340049b7be148635896327a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 29 19:10:12 dist-test-slave-wftq krb5kdc[10686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 19:10:12 dist-test-slave-wftq krb5kdc[10686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.1@KRBTEST.COM
I20260929 19:10:12.291973 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:12.284832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41342 (local address 127.9.236.1:35107)
0929 19:10:12.284892 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:12.284894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:12.285008 (+ 114us) server_negotiation.cc:407] Connection header received
0929 19:10:12.285064 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:12.285065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:12.285074 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:12.285181 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:12.285743 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.286199 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.287051 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:12.287259 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:12.287316 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:12.289614 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:12.289615 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:12.289616 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:12.289623 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:12.290888 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:12.291252 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:12.291253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:12.291253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:12.291300 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:12.291456 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:12.291456 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:12.291457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:12.291535 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:12.291614 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:12.291836 (+ 222us) server_negotiation.cc:299] Negotiation successful
0929 19:10:12.291899 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260929 19:10:12.292483 10836 tablet_service.cc:2908] Rejecting scan request for tablet 2759273b58a444d9997e6052164bf1c8: Uninitialized: clean time has not yet been initialized
I20260929 19:10:12.293606 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 10793
I20260929 19:10:12.298455 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 10927
I20260929 19:10:12.302814 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 11061
I20260929 19:10:12.307551 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 10699
2026-09-29T19:10:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1995 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:10:12 dist-test-slave-wftq krb5kdc[11231](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:10:12 dist-test-slave-wftq krb5kdc[11231](info): set up 2 sockets
Sep 29 19:10:12 dist-test-slave-wftq krb5kdc[11231](info): commencing operation
krb5kdc: starting...
W20260929 19:10:13.265015 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:10:13 dist-test-slave-wftq krb5kdc[11231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:10:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:10:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:10:13.385987 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:41803
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:41803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:13.468235 11244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:13.468443 11244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:13.468467 11244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:13.469656 11244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:13.469707 11244 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 19:10:13.469720 11244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:13.469735 11244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:13.469746 11244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:13.469758 11244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:13.471282 11244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:41803
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:41803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:13.471508 11244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:13.471737 11244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:13.476838 11249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:13.476859 11250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:13.476840 11252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:13.477149 11244 server_base.cc:1034] running on GCE node
I20260929 19:10:13.477403 11244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:13.477648 11244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:13.478843 11244 hybrid_clock.cc:648] HybridClock initialized: now 1790709013478817 us; error 40 us; skew 500 ppm
Sep 29 19:10:13 dist-test-slave-wftq krb5kdc[11231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709013, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:13.481287 11244 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:10:13.481694 11244 webserver.cc:460] Webserver started at http://127.9.236.62:35063/ using document root <none> and password file <none>
I20260929 19:10:13.481876 11244 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:13.481917 11244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:13.482004 11244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:13.482961 11244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "760cd1a69d5d44e5ab5e287c0e286071"
format_stamp: "Formatted at 2026-09-29 19:10:13 on dist-test-slave-wftq"
server_key: "4876965cd1338e3c50425cae8bf26d7e"
server_key_iv: "22f24070efc213df80314e29046bdd81"
server_key_version: "encryptionkey@0"
I20260929 19:10:13.483253 11244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "760cd1a69d5d44e5ab5e287c0e286071"
format_stamp: "Formatted at 2026-09-29 19:10:13 on dist-test-slave-wftq"
server_key: "4876965cd1338e3c50425cae8bf26d7e"
server_key_iv: "22f24070efc213df80314e29046bdd81"
server_key_version: "encryptionkey@0"
I20260929 19:10:13.484504 11244 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:10:13.485249 11259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:13.485432 11244 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:13.485535 11244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "760cd1a69d5d44e5ab5e287c0e286071"
format_stamp: "Formatted at 2026-09-29 19:10:13 on dist-test-slave-wftq"
server_key: "4876965cd1338e3c50425cae8bf26d7e"
server_key_iv: "22f24070efc213df80314e29046bdd81"
server_key_version: "encryptionkey@0"
I20260929 19:10:13.485617 11244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:13.519649 11244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:13.521957 11244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:13.522083 11244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:13.527431 11244 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:41803
I20260929 19:10:13.527539 11321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:41803 every 8 connection(s)
I20260929 19:10:13.527870 11244 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:10:13.528625 11322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:13.530932 11322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071: Bootstrap starting.
I20260929 19:10:13.531309 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 11244
I20260929 19:10:13.531397 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:10:13.531548 10160 external_mini_cluster.cc:1428] Setting key 625cbc76fb19a4167a687684a1d84754
I20260929 19:10:13.531652 11322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:13.531955 11322 log.cc:824] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:13.532763 11322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071: No bootstrap required, opened a new log
I20260929 19:10:13.534617 11322 raft_consensus.cc:348] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "760cd1a69d5d44e5ab5e287c0e286071" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 41803 } }
I20260929 19:10:13.534783 11322 raft_consensus.cc:374] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:13.534806 11322 raft_consensus.cc:729] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 760cd1a69d5d44e5ab5e287c0e286071, State: Initialized, Role: FOLLOWER
I20260929 19:10:13.534933 11322 consensus_queue.cc:260] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "760cd1a69d5d44e5ab5e287c0e286071" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 41803 } }
I20260929 19:10:13.535025 11322 raft_consensus.cc:388] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:10:13.535049 11322 raft_consensus.cc:482] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:10:13.535108 11322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:13.535748 11322 raft_consensus.cc:504] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "760cd1a69d5d44e5ab5e287c0e286071" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 41803 } }
I20260929 19:10:13.535856 11322 leader_election.cc:302] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 760cd1a69d5d44e5ab5e287c0e286071; no voters:
I20260929 19:10:13.536026 11322 leader_election.cc:288] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:10:13.536083 11327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:13.536315 11322 sys_catalog.cc:564] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:13.536350 11327 raft_consensus.cc:686] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071 [term 1 LEADER]: Becoming Leader. State: Replica: 760cd1a69d5d44e5ab5e287c0e286071, State: Running, Role: LEADER
I20260929 19:10:13.536794 11327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "760cd1a69d5d44e5ab5e287c0e286071" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 41803 } }
I20260929 19:10:13.538249 11329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 760cd1a69d5d44e5ab5e287c0e286071. Latest consensus state: current_term: 1 leader_uuid: "760cd1a69d5d44e5ab5e287c0e286071" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "760cd1a69d5d44e5ab5e287c0e286071" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 41803 } } }
I20260929 19:10:13.538269 11328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "760cd1a69d5d44e5ab5e287c0e286071" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "760cd1a69d5d44e5ab5e287c0e286071" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 41803 } } }
I20260929 19:10:13.538342 11329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:13.538369 11328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071 [sys.catalog]: This master's current role is: LEADER
Sep 29 19:10:13 dist-test-slave-wftq krb5kdc[11231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:13.538756 11342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:13.539520 11342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:10:13.541643 11342 catalog_manager.cc:1347] Generated new cluster ID: 5031969c48994480a60477ecf7fae64e
I20260929 19:10:13.541702 11342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:10:13.542712 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:13.532108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40862 (local address 127.9.236.62:41803)
0929 19:10:13.532317 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:13.532326 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:13.532354 (+ 28us) server_negotiation.cc:407] Connection header received
0929 19:10:13.532790 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:13.532796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:13.532854 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:13.532989 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:13.533724 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:13.535068 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:13.535780 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:13.536383 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:13.536483 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:13.539196 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:13.539201 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:13.539202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:13.539215 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:13.541004 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:13.541385 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:13.541388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:13.541389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:13.541478 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:13.541664 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:13.541665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:13.541665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:13.541981 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:13.542126 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:13.542363 (+ 237us) server_negotiation.cc:299] Negotiation successful
0929 19:10:13.542504 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":78,"threads_started":1}
I20260929 19:10:13.550822 11342 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:10:13.551368 11342 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:13.558976 11342 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071: Generated new TSK 0
I20260929 19:10:13.559123 11342 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.236.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
I20260929 19:10:13.592226 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:41803
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:13.693699 11349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:13.693917 11349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:13.693948 11349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:13.695106 11349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:13.695148 11349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:13.695184 11349 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 19:10:13.695209 11349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:10:13.696657 11349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.236.62:41803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:13.696892 11349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:13.697129 11349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:13.702553 11354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:13.702610 11349 server_base.cc:1034] running on GCE node
W20260929 19:10:13.702553 11357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:13.702589 11355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:13.703037 11349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:13.703297 11349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:13.704444 11349 hybrid_clock.cc:648] HybridClock initialized: now 1790709013704423 us; error 42 us; skew 500 ppm
Sep 29 19:10:13 dist-test-slave-wftq krb5kdc[11231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709013, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:13.711383 11349 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:10:13.711839 11349 webserver.cc:460] Webserver started at http://127.9.236.1:34239/ using document root <none> and password file <none>
I20260929 19:10:13.712040 11349 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:13.712085 11349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:13.712201 11349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:13.713290 11349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "c52bd8c1c28746ecad9368f6316baf25"
format_stamp: "Formatted at 2026-09-29 19:10:13 on dist-test-slave-wftq"
server_key: "61785df7972661b7de791b5a79de00cc"
server_key_iv: "c6b843401dd88507b3d69fa1c6800b20"
server_key_version: "encryptionkey@0"
I20260929 19:10:13.713613 11349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "c52bd8c1c28746ecad9368f6316baf25"
format_stamp: "Formatted at 2026-09-29 19:10:13 on dist-test-slave-wftq"
server_key: "61785df7972661b7de791b5a79de00cc"
server_key_iv: "c6b843401dd88507b3d69fa1c6800b20"
server_key_version: "encryptionkey@0"
I20260929 19:10:13.714923 11349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:13.715660 11364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:13.715857 11349 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:10:13.715955 11349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "c52bd8c1c28746ecad9368f6316baf25"
format_stamp: "Formatted at 2026-09-29 19:10:13 on dist-test-slave-wftq"
server_key: "61785df7972661b7de791b5a79de00cc"
server_key_iv: "c6b843401dd88507b3d69fa1c6800b20"
server_key_version: "encryptionkey@0"
I20260929 19:10:13.716035 11349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:13.735334 11349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:13.737587 11349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:13.737711 11349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:13.738329 11349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:13.738379 11349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:13.738404 11349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:13.738417 11349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 19:10:13 dist-test-slave-wftq krb5kdc[11231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709013, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:13.746203 11349 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:33039
I20260929 19:10:13.746629 11479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:33039 every 8 connection(s)
I20260929 19:10:13.746712 11349 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:10:13.748167 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 11349
I20260929 19:10:13.748250 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:10:13.748404 10160 external_mini_cluster.cc:1428] Setting key 4b5277ddbd0c4b9df453317053f42ae6
I20260929 19:10:13.749243 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:13.738873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:42467 (local address 127.9.236.62:41803)
0929 19:10:13.738952 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:13.738955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:13.739186 (+ 231us) server_negotiation.cc:407] Connection header received
0929 19:10:13.739701 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:13.739703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:13.739718 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:13.739828 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:13.740653 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:13.741493 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:13.742940 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:13.743262 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:13.743327 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:13.746006 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:13.746008 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:13.746008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:13.746016 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:13.747630 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:13.748042 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:13.748043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:13.748044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:13.748107 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:13.748345 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:13.748346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:13.748347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:13.748475 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:13.748537 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:13.749049 (+ 512us) server_negotiation.cc:299] Negotiation successful
0929 19:10:13.749158 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 19:10:13.749361 11373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:13.738834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:41803 (local address 127.9.236.1:42467)
0929 19:10:13.739025 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:13.739066 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:13.739591 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:13.739888 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:13.739892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:13.740247 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:13.740533 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:13.740539 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:13.741597 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:13.741597 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:13.742857 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:13.742859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:13.743339 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:13.743339 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:13.743476 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:13.744073 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:13.744081 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:13.745875 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:13.747713 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:13.747720 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:13.747722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:13.747972 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:13.748161 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:13.748162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:13.748162 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:13.748291 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:13.748574 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:13.748582 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:13.748795 (+ 213us) client_negotiation.cc:769] Sending connection context
0929 19:10:13.748917 (+ 122us) client_negotiation.cc:240] Negotiation successful
0929 19:10:13.749121 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":117,"threads_started":1}
I20260929 19:10:13.749886 11480 heartbeater.cc:347] Connected to a master server at 127.9.236.62:41803
I20260929 19:10:13.749979 11480 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:13.750169 11480 heartbeater.cc:511] Master 127.9.236.62:41803 requested a full tablet report, sending...
I20260929 19:10:13.750775 11275 ts_manager.cc:194] Registered new tserver with Master: c52bd8c1c28746ecad9368f6316baf25 (127.9.236.1:33039)
I20260929 19:10:13.751670 11275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:42467
WARNING: no policy specified for kudu/127.9.236.2@KRBTEST.COM; defaulting to no policy
I20260929 19:10:13.758101 11487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:13.752077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:41803 (local address 127.9.236.1:43225)
0929 19:10:13.752262 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:13.752274 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:13.752356 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:13.752515 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:13.752516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:13.752737 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:13.752805 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:13.752807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:13.753579 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:13.753580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:13.754487 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:13.754490 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:13.754835 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:13.754836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:13.754916 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:13.755374 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:13.755379 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:13.755782 (+ 403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:13.757261 (+ 1479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:13.757262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:13.757262 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:13.757416 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:13.757582 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:13.757583 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:13.757583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:13.757628 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:13.757846 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:13.757847 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:13.757884 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 19:10:13.757946 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 19:10:13.758033 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":82,"threads_started":1}
I20260929 19:10:13.758133 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:13.751855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:43225 (local address 127.9.236.62:41803)
0929 19:10:13.751913 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:13.751915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:13.752362 (+ 447us) server_negotiation.cc:407] Connection header received
0929 19:10:13.752425 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:13.752425 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:13.752437 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:13.752495 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:13.752893 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:13.753478 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:13.754577 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:13.754760 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:13.754812 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:13.755865 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:13.755866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:13.755866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:13.755872 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:13.757189 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:13.757479 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:13.757479 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:13.757480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:13.757525 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:13.757708 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:13.757708 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:13.757708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:13.757797 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:13.757845 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:13.757981 (+ 136us) server_negotiation.cc:299] Negotiation successful
0929 19:10:13.758052 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
I20260929 19:10:13.773926 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:41803
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:13.853291 11489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:13.853495 11489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:13.853525 11489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:13.854661 11489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:13.854702 11489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:13.854728 11489 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 19:10:13.854750 11489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:10:13.856103 11489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.236.62:41803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:13.856334 11489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:13.856532 11489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:13.861775 11495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:13.861832 11494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:13.861994 11489 server_base.cc:1034] running on GCE node
W20260929 19:10:13.862111 11497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:13.862352 11489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:13.862620 11489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:13.863792 11489 hybrid_clock.cc:648] HybridClock initialized: now 1790709013863776 us; error 45 us; skew 500 ppm
Sep 29 19:10:13 dist-test-slave-wftq krb5kdc[11231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709013, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:13.866585 11489 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:10:13.867040 11489 webserver.cc:460] Webserver started at http://127.9.236.2:36185/ using document root <none> and password file <none>
I20260929 19:10:13.867239 11489 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:13.867281 11489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:13.867372 11489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:13.868510 11489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "5528277035724bef87204f4738df1788"
format_stamp: "Formatted at 2026-09-29 19:10:13 on dist-test-slave-wftq"
server_key: "3f7b33f8e066f14cafbcb624c62bb86d"
server_key_iv: "4c1bda171c1e6874c2860a2dfb054374"
server_key_version: "encryptionkey@0"
I20260929 19:10:13.868860 11489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "5528277035724bef87204f4738df1788"
format_stamp: "Formatted at 2026-09-29 19:10:13 on dist-test-slave-wftq"
server_key: "3f7b33f8e066f14cafbcb624c62bb86d"
server_key_iv: "4c1bda171c1e6874c2860a2dfb054374"
server_key_version: "encryptionkey@0"
I20260929 19:10:13.870349 11489 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260929 19:10:13.871199 11504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:13.871469 11489 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:13.871552 11489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "5528277035724bef87204f4738df1788"
format_stamp: "Formatted at 2026-09-29 19:10:13 on dist-test-slave-wftq"
server_key: "3f7b33f8e066f14cafbcb624c62bb86d"
server_key_iv: "4c1bda171c1e6874c2860a2dfb054374"
server_key_version: "encryptionkey@0"
I20260929 19:10:13.871622 11489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:13.879120 11489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:13.880882 11489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:13.881014 11489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:13.881537 11489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:13.881572 11489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:13.881598 11489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:13.881626 11489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 19:10:13 dist-test-slave-wftq krb5kdc[11231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709013, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:13.889475 11489 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:36815
I20260929 19:10:13.889590 11619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:36815 every 8 connection(s)
I20260929 19:10:13.889890 11489 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:10:13.891979 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:13.882107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:34527 (local address 127.9.236.62:41803)
0929 19:10:13.882157 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:13.882160 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:13.882392 (+ 232us) server_negotiation.cc:407] Connection header received
0929 19:10:13.882997 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:13.882999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:13.883017 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:13.883099 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:13.883978 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:13.884752 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:13.886196 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:13.886505 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:13.886576 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:13.889457 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:13.889459 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:13.889459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:13.889468 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:13.890877 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:13.891231 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:13.891231 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:13.891232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:13.891275 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:13.891464 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:13.891464 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:13.891464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:13.891569 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:13.891620 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:13.891853 (+ 233us) server_negotiation.cc:299] Negotiation successful
0929 19:10:13.891909 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 19:10:13.892150 11513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:13.882089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:41803 (local address 127.9.236.2:34527)
0929 19:10:13.882270 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:13.882310 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:13.882865 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:13.883165 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:13.883170 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:13.883527 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:13.883875 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:13.883881 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:13.884848 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:13.884849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:13.886100 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:13.886103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:13.886617 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:13.886618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:13.886745 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:13.887255 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:13.887266 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:13.889336 (+ 2070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:13.890950 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:13.890953 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:13.890955 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:13.891166 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:13.891324 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:13.891325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:13.891325 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:13.891411 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:13.891617 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:13.891620 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:13.891728 (+ 108us) client_negotiation.cc:769] Sending connection context
0929 19:10:13.891833 (+ 105us) client_negotiation.cc:240] Negotiation successful
0929 19:10:13.891952 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":97,"threads_started":1}
I20260929 19:10:13.892640 11620 heartbeater.cc:347] Connected to a master server at 127.9.236.62:41803
I20260929 19:10:13.892725 11620 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:13.892921 11620 heartbeater.cc:511] Master 127.9.236.62:41803 requested a full tablet report, sending...
I20260929 19:10:13.893383 11276 ts_manager.cc:194] Registered new tserver with Master: 5528277035724bef87204f4738df1788 (127.9.236.2:36815)
I20260929 19:10:13.893831 11276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:34527
I20260929 19:10:13.898731 11626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:13.893686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:41803 (local address 127.9.236.2:57947)
0929 19:10:13.893845 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:13.893859 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:13.893949 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:13.894118 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:13.894120 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:13.894346 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:13.894417 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:13.894419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:13.895088 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:13.895088 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:13.895572 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:13.895574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:13.895915 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:13.895916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:13.895967 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:13.896343 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:13.896348 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:13.896602 (+ 254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:13.897863 (+ 1261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:13.897863 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:13.897864 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:13.898018 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:13.898180 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:13.898180 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:13.898180 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:13.898226 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:13.898464 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:13.898465 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:13.898498 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 19:10:13.898545 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 19:10:13.898614 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":74,"threads_started":1}
I20260929 19:10:13.898768 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:13.893751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:57947 (local address 127.9.236.62:41803)
0929 19:10:13.893794 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:13.893795 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:13.893982 (+ 187us) server_negotiation.cc:407] Connection header received
0929 19:10:13.894017 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:13.894018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:13.894032 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:13.894105 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:13.894507 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:13.894989 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:13.895663 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:13.895846 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:13.895913 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:13.896679 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:13.896680 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:13.896680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:13.896685 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:13.897778 (+ 1093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:13.898085 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:13.898085 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:13.898085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:13.898128 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:13.898274 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:13.898275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:13.898275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:13.898402 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:13.898464 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:13.898578 (+ 114us) server_negotiation.cc:299] Negotiation successful
0929 19:10:13.898651 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 19:10:13.898916 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 11489
I20260929 19:10:13.899012 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:10:13.899175 10160 external_mini_cluster.cc:1428] Setting key 155119d2ca4cdb6685969c0eec019247
WARNING: no policy specified for kudu/127.9.236.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
I20260929 19:10:13.923260 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:41803
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:14.001820 11629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:14.002022 11629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:14.002050 11629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:14.003187 11629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:14.003229 11629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:14.003257 11629 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 19:10:14.003279 11629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:10:14.004680 11629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.236.62:41803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:14.004904 11629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:14.005131 11629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:14.010388 11637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:14.010438 11634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:14.010388 11635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:14.010667 11629 server_base.cc:1034] running on GCE node
I20260929 19:10:14.010998 11629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:14.011235 11629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:14.012387 11629 hybrid_clock.cc:648] HybridClock initialized: now 1790709014012363 us; error 42 us; skew 500 ppm
Sep 29 19:10:14 dist-test-slave-wftq krb5kdc[11231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709014, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:14.015086 11629 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:10:14.015586 11629 webserver.cc:460] Webserver started at http://127.9.236.3:37197/ using document root <none> and password file <none>
I20260929 19:10:14.015817 11629 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:14.015864 11629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:14.015990 11629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:14.017127 11629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "6783f18b0a934955aa840df9dbfc2a1c"
format_stamp: "Formatted at 2026-09-29 19:10:14 on dist-test-slave-wftq"
server_key: "9c50ff1c4cad4697e6bfddf8125f643a"
server_key_iv: "8f33f985d42f7390d7e1353ff9b36b26"
server_key_version: "encryptionkey@0"
I20260929 19:10:14.017494 11629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "6783f18b0a934955aa840df9dbfc2a1c"
format_stamp: "Formatted at 2026-09-29 19:10:14 on dist-test-slave-wftq"
server_key: "9c50ff1c4cad4697e6bfddf8125f643a"
server_key_iv: "8f33f985d42f7390d7e1353ff9b36b26"
server_key_version: "encryptionkey@0"
I20260929 19:10:14.018852 11629 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:14.019637 11644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:14.019842 11629 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:14.019922 11629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "6783f18b0a934955aa840df9dbfc2a1c"
format_stamp: "Formatted at 2026-09-29 19:10:14 on dist-test-slave-wftq"
server_key: "9c50ff1c4cad4697e6bfddf8125f643a"
server_key_iv: "8f33f985d42f7390d7e1353ff9b36b26"
server_key_version: "encryptionkey@0"
I20260929 19:10:14.019994 11629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:14.038683 11629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:14.040774 11629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:14.040894 11629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:14.041492 11629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:14.041539 11629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:14.041571 11629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:14.041586 11629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 19:10:14 dist-test-slave-wftq krb5kdc[11231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709014, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:14.049463 11629 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:45121
I20260929 19:10:14.049515 11759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:45121 every 8 connection(s)
I20260929 19:10:14.049912 11629 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:10:14.052011 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.042040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:35569 (local address 127.9.236.62:41803)
0929 19:10:14.042115 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:14.042118 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:14.042339 (+ 221us) server_negotiation.cc:407] Connection header received
0929 19:10:14.042867 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:14.042868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:14.042879 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:14.042931 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:14.043769 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.044540 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.046044 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.046338 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.046414 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.049229 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:14.049231 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:14.049231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:14.049239 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:14.050730 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:14.051147 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:14.051148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:14.051148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:14.051202 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:14.051417 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:14.051418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:14.051418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:14.051537 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:14.051593 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:14.051843 (+ 250us) server_negotiation.cc:299] Negotiation successful
0929 19:10:14.051917 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 19:10:14.052208 11653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.042015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:41803 (local address 127.9.236.3:35569)
0929 19:10:14.042198 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:14.042240 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:14.042754 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:14.043009 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:14.043012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:14.043363 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:14.043643 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.043648 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.044658 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.044659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.045961 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.045964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.046456 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.046458 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.046668 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.047218 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:14.047229 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:14.049130 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:14.050804 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:14.050811 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:14.050814 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:14.051081 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:14.051260 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:14.051261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:14.051261 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:14.051364 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:14.051590 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:14.051596 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:14.051731 (+ 135us) client_negotiation.cc:769] Sending connection context
0929 19:10:14.051817 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 19:10:14.051981 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":108,"threads_started":1}
I20260929 19:10:14.052856 11760 heartbeater.cc:347] Connected to a master server at 127.9.236.62:41803
I20260929 19:10:14.052949 11760 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:14.053185 11760 heartbeater.cc:511] Master 127.9.236.62:41803 requested a full tablet report, sending...
I20260929 19:10:14.053613 11275 ts_manager.cc:194] Registered new tserver with Master: 6783f18b0a934955aa840df9dbfc2a1c (127.9.236.3:45121)
I20260929 19:10:14.054051 11275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:35569
I20260929 19:10:14.058390 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 11629
I20260929 19:10:14.058470 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:10:14.058625 10160 external_mini_cluster.cc:1428] Setting key b67ad53666876cbdcc95f7d238754e10
I20260929 19:10:14.059574 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:10:14.059865 11766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.054035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:41803 (local address 127.9.236.3:51991)
0929 19:10:14.054300 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:14.054310 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:14.054368 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:14.054512 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:14.054513 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:14.054738 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:14.054806 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.054809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.055509 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.055510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.056054 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.056056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.056408 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.056409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.056496 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.056928 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:14.056932 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:14.057284 (+ 352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:14.058854 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:14.058855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:14.058856 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:14.059062 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:14.059251 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:14.059252 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:14.059252 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:14.059321 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:14.059583 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:14.059584 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:14.059631 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 19:10:14.059694 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 19:10:14.059793 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":90,"threads_started":1}
I20260929 19:10:14.059962 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.054133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:51991 (local address 127.9.236.62:41803)
0929 19:10:14.054186 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:14.054187 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:14.054368 (+ 181us) server_negotiation.cc:407] Connection header received
0929 19:10:14.054430 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:14.054431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:14.054443 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:14.054504 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:14.054880 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.055404 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.056129 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.056336 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.056401 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.057393 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:14.057395 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:14.057395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:14.057402 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:14.058758 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:14.059116 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:14.059117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:14.059117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:14.059183 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:14.059379 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:14.059379 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:14.059380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:14.059521 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:14.059581 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:14.059773 (+ 192us) server_negotiation.cc:299] Negotiation successful
0929 19:10:14.059875 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
Sep 29 19:10:14 dist-test-slave-wftq krb5kdc[11231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709014, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 19:10:14 dist-test-slave-wftq krb5kdc[11231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709014, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:14.082749 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.075665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40876 (local address 127.9.236.62:41803)
0929 19:10:14.075740 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:14.075742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:14.075751 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:14.075778 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:14.075779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:14.075789 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:14.075859 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:14.076458 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.076970 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.077832 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.078022 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.078091 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.080522 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:14.080523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:14.080523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:14.080531 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:14.081799 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:14.082132 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:14.082132 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:14.082133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:14.082179 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:14.082323 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:14.082323 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:14.082323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:14.082436 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:14.082513 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:14.082600 (+ 87us) server_negotiation.cc:299] Negotiation successful
0929 19:10:14.082654 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 19:10:14.084368 11275 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:10:14.086112 11275 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:10:14.092850 11778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.088497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:33039 (local address 127.0.0.1:36634)
0929 19:10:14.088725 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:14.089039 (+ 314us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:14.089084 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:14.089523 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:14.089524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:14.089526 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:14.089572 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.089574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.090485 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.090486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.091566 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.091569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.092552 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.092553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.092629 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.092646 (+ 17us) client_negotiation.cc:769] Sending connection context
0929 19:10:14.092723 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 19:10:14.092775 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":62,"threads_started":1}
I20260929 19:10:14.092948 11781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.088735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36634 (local address 127.9.236.1:33039)
0929 19:10:14.089313 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:14.089318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:14.089326 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:10:14.089361 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:14.089363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:14.089380 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:14.089451 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:14.089625 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.090414 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.091679 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.092469 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.092546 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.092610 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:14.092794 (+ 184us) server_negotiation.cc:299] Negotiation successful
0929 19:10:14.092860 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":104,"threads_started":1}
I20260929 19:10:14.093242 11777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.088502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:45121 (local address 127.0.0.1:49400)
0929 19:10:14.088826 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:14.088861 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:14.088931 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:14.089440 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:14.089441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:14.089445 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:14.089519 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.089522 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.091008 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.091009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.092014 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.092015 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.092952 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.092953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.093034 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.093039 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:10:14.093066 (+ 27us) client_negotiation.cc:240] Negotiation successful
0929 19:10:14.093149 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":3,"thread_start_us":130,"threads_started":1}
I20260929 19:10:14.093685 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.088639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49400 (local address 127.9.236.3:45121)
0929 19:10:14.089225 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:14.089229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:14.089239 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:10:14.089280 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:14.089282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:14.089297 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:14.089638 (+ 341us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:14.089699 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.090917 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.092096 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.092858 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.093449 (+ 591us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.093505 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:14.093529 (+ 24us) server_negotiation.cc:299] Negotiation successful
0929 19:10:14.093566 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":104,"threads_started":1}
I20260929 19:10:14.093899 11414 tablet_service.cc:1467] Processing CreateTablet for tablet 30051f9bf2d94b03b29470d4d287bdbd (DEFAULT_TABLE table=test-table [id=cb736d4107a54ca2b8151378c6fd5a10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:14.094295 11780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.088750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:36815 (local address 127.0.0.1:38850)
0929 19:10:14.089055 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:14.089063 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:14.089157 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:14.089407 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:14.089410 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:14.089413 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:14.089492 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.089495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.091684 (+ 2189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.091685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.092627 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.092629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.094065 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.094066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.094131 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.094135 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:10:14.094177 (+ 42us) client_negotiation.cc:240] Negotiation successful
0929 19:10:14.094223 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":3,"thread_start_us":77,"threads_started":1}
I20260929 19:10:14.094296 11414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30051f9bf2d94b03b29470d4d287bdbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:14.094578 11782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.088873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38850 (local address 127.9.236.2:36815)
0929 19:10:14.089073 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:14.089078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:14.089127 (+ 49us) server_negotiation.cc:407] Connection header received
0929 19:10:14.089250 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:14.089253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:14.089273 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:14.089379 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:14.089579 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.090905 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.092750 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.093906 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.093984 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.094033 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:14.094454 (+ 421us) server_negotiation.cc:299] Negotiation successful
0929 19:10:14.094492 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":98,"threads_started":1}
I20260929 19:10:14.095237 11554 tablet_service.cc:1467] Processing CreateTablet for tablet 30051f9bf2d94b03b29470d4d287bdbd (DEFAULT_TABLE table=test-table [id=cb736d4107a54ca2b8151378c6fd5a10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:14.095562 11554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30051f9bf2d94b03b29470d4d287bdbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:14.097584 11783 tablet_bootstrap.cc:492] T 30051f9bf2d94b03b29470d4d287bdbd P c52bd8c1c28746ecad9368f6316baf25: Bootstrap starting.
I20260929 19:10:14.097692 11694 tablet_service.cc:1467] Processing CreateTablet for tablet 30051f9bf2d94b03b29470d4d287bdbd (DEFAULT_TABLE table=test-table [id=cb736d4107a54ca2b8151378c6fd5a10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:14.098033 11694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30051f9bf2d94b03b29470d4d287bdbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:14.098153 11783 tablet_bootstrap.cc:654] T 30051f9bf2d94b03b29470d4d287bdbd P c52bd8c1c28746ecad9368f6316baf25: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:14.098433 11783 log.cc:824] T 30051f9bf2d94b03b29470d4d287bdbd P c52bd8c1c28746ecad9368f6316baf25: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:14.099107 11783 tablet_bootstrap.cc:492] T 30051f9bf2d94b03b29470d4d287bdbd P c52bd8c1c28746ecad9368f6316baf25: No bootstrap required, opened a new log
I20260929 19:10:14.099157 11785 tablet_bootstrap.cc:492] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788: Bootstrap starting.
I20260929 19:10:14.099215 11783 ts_tablet_manager.cc:1397] T 30051f9bf2d94b03b29470d4d287bdbd P c52bd8c1c28746ecad9368f6316baf25: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:10:14.099872 11785 tablet_bootstrap.cc:654] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:14.100236 11785 log.cc:824] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:14.101012 11787 tablet_bootstrap.cc:492] T 30051f9bf2d94b03b29470d4d287bdbd P 6783f18b0a934955aa840df9dbfc2a1c: Bootstrap starting.
I20260929 19:10:14.101076 11783 raft_consensus.cc:348] T 30051f9bf2d94b03b29470d4d287bdbd P c52bd8c1c28746ecad9368f6316baf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } }
I20260929 19:10:14.101259 11783 raft_consensus.cc:374] T 30051f9bf2d94b03b29470d4d287bdbd P c52bd8c1c28746ecad9368f6316baf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:14.101292 11783 raft_consensus.cc:729] T 30051f9bf2d94b03b29470d4d287bdbd P c52bd8c1c28746ecad9368f6316baf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c52bd8c1c28746ecad9368f6316baf25, State: Initialized, Role: FOLLOWER
I20260929 19:10:14.101402 11783 consensus_queue.cc:260] T 30051f9bf2d94b03b29470d4d287bdbd P c52bd8c1c28746ecad9368f6316baf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } }
I20260929 19:10:14.101657 11783 ts_tablet_manager.cc:1428] T 30051f9bf2d94b03b29470d4d287bdbd P c52bd8c1c28746ecad9368f6316baf25: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:10:14.101737 11480 heartbeater.cc:503] Master 127.9.236.62:41803 was elected leader, sending a full tablet report...
I20260929 19:10:14.101958 11787 tablet_bootstrap.cc:654] T 30051f9bf2d94b03b29470d4d287bdbd P 6783f18b0a934955aa840df9dbfc2a1c: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:14.102226 11785 tablet_bootstrap.cc:492] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788: No bootstrap required, opened a new log
I20260929 19:10:14.102288 11785 ts_tablet_manager.cc:1397] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260929 19:10:14.102317 11787 log.cc:824] T 30051f9bf2d94b03b29470d4d287bdbd P 6783f18b0a934955aa840df9dbfc2a1c: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:14.103173 11787 tablet_bootstrap.cc:492] T 30051f9bf2d94b03b29470d4d287bdbd P 6783f18b0a934955aa840df9dbfc2a1c: No bootstrap required, opened a new log
I20260929 19:10:14.103235 11787 ts_tablet_manager.cc:1397] T 30051f9bf2d94b03b29470d4d287bdbd P 6783f18b0a934955aa840df9dbfc2a1c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:10:14.103981 11785 raft_consensus.cc:348] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } }
I20260929 19:10:14.104102 11785 raft_consensus.cc:374] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:14.104130 11785 raft_consensus.cc:729] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5528277035724bef87204f4738df1788, State: Initialized, Role: FOLLOWER
I20260929 19:10:14.104235 11785 consensus_queue.cc:260] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } }
I20260929 19:10:14.104462 11785 ts_tablet_manager.cc:1428] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:10:14.104552 11620 heartbeater.cc:503] Master 127.9.236.62:41803 was elected leader, sending a full tablet report...
I20260929 19:10:14.104836 11787 raft_consensus.cc:348] T 30051f9bf2d94b03b29470d4d287bdbd P 6783f18b0a934955aa840df9dbfc2a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } }
I20260929 19:10:14.105008 11787 raft_consensus.cc:374] T 30051f9bf2d94b03b29470d4d287bdbd P 6783f18b0a934955aa840df9dbfc2a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:14.105067 11787 raft_consensus.cc:729] T 30051f9bf2d94b03b29470d4d287bdbd P 6783f18b0a934955aa840df9dbfc2a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6783f18b0a934955aa840df9dbfc2a1c, State: Initialized, Role: FOLLOWER
I20260929 19:10:14.105202 11787 consensus_queue.cc:260] T 30051f9bf2d94b03b29470d4d287bdbd P 6783f18b0a934955aa840df9dbfc2a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } }
I20260929 19:10:14.105463 11787 ts_tablet_manager.cc:1428] T 30051f9bf2d94b03b29470d4d287bdbd P 6783f18b0a934955aa840df9dbfc2a1c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:10:14.105475 11760 heartbeater.cc:503] Master 127.9.236.62:41803 was elected leader, sending a full tablet report...
W20260929 19:10:14.140532 11621 tablet.cc:2369] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:10:14.193900 11790 raft_consensus.cc:482] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:10:14.194124 11790 raft_consensus.cc:504] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } }
I20260929 19:10:14.194515 11790 leader_election.cc:288] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6783f18b0a934955aa840df9dbfc2a1c (127.9.236.3:45121), c52bd8c1c28746ecad9368f6316baf25 (127.9.236.1:33039)
I20260929 19:10:14.197726 11781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.194916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:36383 (local address 127.9.236.1:33039)
0929 19:10:14.194984 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:14.194986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:14.195095 (+ 109us) server_negotiation.cc:407] Connection header received
0929 19:10:14.195158 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:14.195160 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:14.195175 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:14.195238 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:14.195429 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.195944 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.196951 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.197371 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.197442 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.197464 (+ 22us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:14.197608 (+ 144us) server_negotiation.cc:299] Negotiation successful
0929 19:10:14.197652 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 19:10:14.197729 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.194859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:43149 (local address 127.9.236.3:45121)
0929 19:10:14.194905 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:14.194907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:14.194919 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:10:14.195049 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:14.195051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:14.195066 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:14.195129 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:14.195329 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.195878 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.196906 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.197354 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.197414 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.197426 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:14.197593 (+ 167us) server_negotiation.cc:299] Negotiation successful
0929 19:10:14.197629 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 19:10:14.197762 11792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.194745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:33039 (local address 127.9.236.2:36383)
0929 19:10:14.195004 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:14.195016 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:14.195095 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:14.195252 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:14.195252 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:14.195254 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:14.195356 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.195358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.196030 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.196031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.196849 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.196852 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.197462 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.197463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.197529 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.197533 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:10:14.197589 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 19:10:14.197637 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":89,"thread_start_us":110,"threads_started":1}
I20260929 19:10:14.197774 11513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.194745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:45121 (local address 127.9.236.2:43149)
0929 19:10:14.194814 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:14.194830 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:14.194932 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:14.195149 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:14.195150 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:14.195152 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:14.195245 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.195247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.196003 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.196004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.196807 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.196809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.197434 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.197434 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.197512 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.197517 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:10:14.197565 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 19:10:14.197620 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 19:10:14.198191 11434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30051f9bf2d94b03b29470d4d287bdbd" candidate_uuid: "5528277035724bef87204f4738df1788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c52bd8c1c28746ecad9368f6316baf25" is_pre_election: true
I20260929 19:10:14.198282 11712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30051f9bf2d94b03b29470d4d287bdbd" candidate_uuid: "5528277035724bef87204f4738df1788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6783f18b0a934955aa840df9dbfc2a1c" is_pre_election: true
I20260929 19:10:14.198359 11434 raft_consensus.cc:2454] T 30051f9bf2d94b03b29470d4d287bdbd P c52bd8c1c28746ecad9368f6316baf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5528277035724bef87204f4738df1788 in term 0.
I20260929 19:10:14.198468 11712 raft_consensus.cc:2454] T 30051f9bf2d94b03b29470d4d287bdbd P 6783f18b0a934955aa840df9dbfc2a1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5528277035724bef87204f4738df1788 in term 0.
I20260929 19:10:14.198601 11508 leader_election.cc:302] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5528277035724bef87204f4738df1788, c52bd8c1c28746ecad9368f6316baf25; no voters:
I20260929 19:10:14.198777 11790 raft_consensus.cc:2781] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:10:14.198856 11790 raft_consensus.cc:482] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:10:14.198880 11790 raft_consensus.cc:3037] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:14.199978 11790 raft_consensus.cc:504] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } }
I20260929 19:10:14.200119 11790 leader_election.cc:288] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 [CANDIDATE]: Term 1 election: Requested vote from peers 6783f18b0a934955aa840df9dbfc2a1c (127.9.236.3:45121), c52bd8c1c28746ecad9368f6316baf25 (127.9.236.1:33039)
I20260929 19:10:14.200340 11434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30051f9bf2d94b03b29470d4d287bdbd" candidate_uuid: "5528277035724bef87204f4738df1788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c52bd8c1c28746ecad9368f6316baf25"
I20260929 19:10:14.200337 11712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30051f9bf2d94b03b29470d4d287bdbd" candidate_uuid: "5528277035724bef87204f4738df1788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6783f18b0a934955aa840df9dbfc2a1c"
I20260929 19:10:14.200425 11434 raft_consensus.cc:3037] T 30051f9bf2d94b03b29470d4d287bdbd P c52bd8c1c28746ecad9368f6316baf25 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:14.200440 11712 raft_consensus.cc:3037] T 30051f9bf2d94b03b29470d4d287bdbd P 6783f18b0a934955aa840df9dbfc2a1c [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:14.201257 11434 raft_consensus.cc:2454] T 30051f9bf2d94b03b29470d4d287bdbd P c52bd8c1c28746ecad9368f6316baf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5528277035724bef87204f4738df1788 in term 1.
I20260929 19:10:14.201324 11712 raft_consensus.cc:2454] T 30051f9bf2d94b03b29470d4d287bdbd P 6783f18b0a934955aa840df9dbfc2a1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5528277035724bef87204f4738df1788 in term 1.
I20260929 19:10:14.201455 11508 leader_election.cc:302] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5528277035724bef87204f4738df1788, c52bd8c1c28746ecad9368f6316baf25; no voters:
I20260929 19:10:14.201556 11790 raft_consensus.cc:2781] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:14.201750 11790 raft_consensus.cc:686] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 [term 1 LEADER]: Becoming Leader. State: Replica: 5528277035724bef87204f4738df1788, State: Running, Role: LEADER
I20260929 19:10:14.201843 11790 consensus_queue.cc:237] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } }
I20260929 19:10:14.202673 11276 catalog_manager.cc:5495] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5528277035724bef87204f4738df1788 (127.9.236.2). New cstate: current_term: 1 leader_uuid: "5528277035724bef87204f4738df1788" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } health_report { overall_health: UNKNOWN } } }
I20260929 19:10:14.209062 11782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.206675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38860 (local address 127.9.236.2:36815)
0929 19:10:14.206742 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:14.206743 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:14.206751 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:10:14.206777 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:14.206778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:14.206795 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:14.206848 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:14.207047 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.207526 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.208293 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.208501 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.208554 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.208743 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:14.208842 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:14.208909 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:14.208948 (+ 39us) server_negotiation.cc:299] Negotiation successful
0929 19:10:14.208971 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 19:10:14.211040 11712 raft_consensus.cc:1261] T 30051f9bf2d94b03b29470d4d287bdbd P 6783f18b0a934955aa840df9dbfc2a1c [term 1 FOLLOWER]: Refusing update from remote peer 5528277035724bef87204f4738df1788: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:14.211047 11434 raft_consensus.cc:1261] T 30051f9bf2d94b03b29470d4d287bdbd P c52bd8c1c28746ecad9368f6316baf25 [term 1 FOLLOWER]: Refusing update from remote peer 5528277035724bef87204f4738df1788: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:14.211412 11790 consensus_queue.cc:1035] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:14.211588 11793 consensus_queue.cc:1035] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:14.214222 11797 mvcc.cc:205] Tried to move back new op lower bound from 7334744122205405184 to 7334744122171445248. Current Snapshot: MvccSnapshot[applied={T|T < 7334744122205405184}]
I20260929 19:10:14.214262 11796 mvcc.cc:205] Tried to move back new op lower bound from 7334744122205405184 to 7334744122171445248. Current Snapshot: MvccSnapshot[applied={T|T < 7334744122205405184}]
I20260929 19:10:14.214900 11798 mvcc.cc:205] Tried to move back new op lower bound from 7334744122205405184 to 7334744122171445248. Current Snapshot: MvccSnapshot[applied={T|T < 7334744122205405184}]
I20260929 19:10:14.218513 11276 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:14.218619 11276 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:14.219563 11276 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071: Sending DeleteTablet for 3 replicas of tablet 30051f9bf2d94b03b29470d4d287bdbd
I20260929 19:10:14.220016 11414 tablet_service.cc:1514] Processing DeleteTablet for tablet 30051f9bf2d94b03b29470d4d287bdbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:14 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:36634
I20260929 19:10:14.220017 11694 tablet_service.cc:1514] Processing DeleteTablet for tablet 30051f9bf2d94b03b29470d4d287bdbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:14 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:49400
I20260929 19:10:14.220207 11276 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:10:14.220366 11276 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:10:14.220332 11554 tablet_service.cc:1514] Processing DeleteTablet for tablet 30051f9bf2d94b03b29470d4d287bdbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:14 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:38850
I20260929 19:10:14.220481 11808 tablet_replica.cc:330] T 30051f9bf2d94b03b29470d4d287bdbd P 6783f18b0a934955aa840df9dbfc2a1c: stopping tablet replica
I20260929 19:10:14.220604 11808 raft_consensus.cc:2229] T 30051f9bf2d94b03b29470d4d287bdbd P 6783f18b0a934955aa840df9dbfc2a1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:14.220738 11808 raft_consensus.cc:2258] T 30051f9bf2d94b03b29470d4d287bdbd P 6783f18b0a934955aa840df9dbfc2a1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:14.220868 11809 tablet_replica.cc:330] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788: stopping tablet replica
I20260929 19:10:14.221017 11809 raft_consensus.cc:2229] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:10:14.221186 11809 raft_consensus.cc:2258] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:14.221508 11808 ts_tablet_manager.cc:1905] T 30051f9bf2d94b03b29470d4d287bdbd P 6783f18b0a934955aa840df9dbfc2a1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:14.222172 11694 tablet_service.cc:1467] Processing CreateTablet for tablet ba398039d4024d7ca3bf910af8e86172 (DEFAULT_TABLE table=test-table [id=b7fd6dbfd7bc421fada7e63006af9940]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:14.222301 11694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba398039d4024d7ca3bf910af8e86172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:14.223595 11787 tablet_bootstrap.cc:492] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c: Bootstrap starting.
I20260929 19:10:14.223948 11414 tablet_service.cc:1467] Processing CreateTablet for tablet ba398039d4024d7ca3bf910af8e86172 (DEFAULT_TABLE table=test-table [id=b7fd6dbfd7bc421fada7e63006af9940]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:14.221863 11554 tablet_service.cc:1467] Processing CreateTablet for tablet ba398039d4024d7ca3bf910af8e86172 (DEFAULT_TABLE table=test-table [id=b7fd6dbfd7bc421fada7e63006af9940]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:14.224134 11787 tablet_bootstrap.cc:654] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:14.224184 11554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba398039d4024d7ca3bf910af8e86172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:14.224751 11787 tablet_bootstrap.cc:492] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c: No bootstrap required, opened a new log
I20260929 19:10:14.224800 11787 ts_tablet_manager.cc:1397] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260929 19:10:14.224984 11787 raft_consensus.cc:348] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } }
I20260929 19:10:14.225051 11787 raft_consensus.cc:374] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:14.225073 11787 raft_consensus.cc:729] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6783f18b0a934955aa840df9dbfc2a1c, State: Initialized, Role: FOLLOWER
I20260929 19:10:14.225153 11787 consensus_queue.cc:260] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } }
I20260929 19:10:14.225261 11787 ts_tablet_manager.cc:1428] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260929 19:10:14.225458 11808 ts_tablet_manager.cc:1918] T 30051f9bf2d94b03b29470d4d287bdbd P 6783f18b0a934955aa840df9dbfc2a1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:10:14.225531 11808 log.cc:1196] T 30051f9bf2d94b03b29470d4d287bdbd P 6783f18b0a934955aa840df9dbfc2a1c: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/30051f9bf2d94b03b29470d4d287bdbd
I20260929 19:10:14.225816 11808 ts_tablet_manager.cc:1939] T 30051f9bf2d94b03b29470d4d287bdbd P 6783f18b0a934955aa840df9dbfc2a1c: Deleting consensus metadata
I20260929 19:10:14.226073 11785 tablet_bootstrap.cc:492] T ba398039d4024d7ca3bf910af8e86172 P 5528277035724bef87204f4738df1788: Bootstrap starting.
I20260929 19:10:14.226321 11261 catalog_manager.cc:4841] TS 6783f18b0a934955aa840df9dbfc2a1c (127.9.236.3:45121): tablet 30051f9bf2d94b03b29470d4d287bdbd (table test-table [id=cb736d4107a54ca2b8151378c6fd5a10]) successfully deleted
I20260929 19:10:14.226480 11785 tablet_bootstrap.cc:654] T ba398039d4024d7ca3bf910af8e86172 P 5528277035724bef87204f4738df1788: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:14.226511 11807 tablet_replica.cc:330] T 30051f9bf2d94b03b29470d4d287bdbd P c52bd8c1c28746ecad9368f6316baf25: stopping tablet replica
I20260929 19:10:14.226567 11414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba398039d4024d7ca3bf910af8e86172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:14.226648 11807 raft_consensus.cc:2229] T 30051f9bf2d94b03b29470d4d287bdbd P c52bd8c1c28746ecad9368f6316baf25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:14.226774 11807 raft_consensus.cc:2258] T 30051f9bf2d94b03b29470d4d287bdbd P c52bd8c1c28746ecad9368f6316baf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:14.227262 11809 ts_tablet_manager.cc:1905] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:14.227305 11807 ts_tablet_manager.cc:1905] T 30051f9bf2d94b03b29470d4d287bdbd P c52bd8c1c28746ecad9368f6316baf25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:14.228183 11783 tablet_bootstrap.cc:492] T ba398039d4024d7ca3bf910af8e86172 P c52bd8c1c28746ecad9368f6316baf25: Bootstrap starting.
I20260929 19:10:14.228636 11783 tablet_bootstrap.cc:654] T ba398039d4024d7ca3bf910af8e86172 P c52bd8c1c28746ecad9368f6316baf25: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:14.229120 11785 tablet_bootstrap.cc:492] T ba398039d4024d7ca3bf910af8e86172 P 5528277035724bef87204f4738df1788: No bootstrap required, opened a new log
I20260929 19:10:14.229223 11809 ts_tablet_manager.cc:1918] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:10:14.229220 11785 ts_tablet_manager.cc:1397] T ba398039d4024d7ca3bf910af8e86172 P 5528277035724bef87204f4738df1788: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260929 19:10:14.229454 11809 log.cc:1196] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/30051f9bf2d94b03b29470d4d287bdbd
I20260929 19:10:14.229535 11785 raft_consensus.cc:348] T ba398039d4024d7ca3bf910af8e86172 P 5528277035724bef87204f4738df1788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } }
I20260929 19:10:14.229627 11785 raft_consensus.cc:374] T ba398039d4024d7ca3bf910af8e86172 P 5528277035724bef87204f4738df1788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:14.229633 11807 ts_tablet_manager.cc:1918] T 30051f9bf2d94b03b29470d4d287bdbd P c52bd8c1c28746ecad9368f6316baf25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:10:14.229678 11785 raft_consensus.cc:729] T ba398039d4024d7ca3bf910af8e86172 P 5528277035724bef87204f4738df1788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5528277035724bef87204f4738df1788, State: Initialized, Role: FOLLOWER
I20260929 19:10:14.229701 11807 log.cc:1196] T 30051f9bf2d94b03b29470d4d287bdbd P c52bd8c1c28746ecad9368f6316baf25: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-0/wal/wals/30051f9bf2d94b03b29470d4d287bdbd
I20260929 19:10:14.229744 11785 consensus_queue.cc:260] T ba398039d4024d7ca3bf910af8e86172 P 5528277035724bef87204f4738df1788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } }
I20260929 19:10:14.229843 11785 ts_tablet_manager.cc:1428] T ba398039d4024d7ca3bf910af8e86172 P 5528277035724bef87204f4738df1788: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:14.229948 11807 ts_tablet_manager.cc:1939] T 30051f9bf2d94b03b29470d4d287bdbd P c52bd8c1c28746ecad9368f6316baf25: Deleting consensus metadata
I20260929 19:10:14.230119 11809 ts_tablet_manager.cc:1939] T 30051f9bf2d94b03b29470d4d287bdbd P 5528277035724bef87204f4738df1788: Deleting consensus metadata
I20260929 19:10:14.230382 11263 catalog_manager.cc:4841] TS c52bd8c1c28746ecad9368f6316baf25 (127.9.236.1:33039): tablet 30051f9bf2d94b03b29470d4d287bdbd (table test-table [id=cb736d4107a54ca2b8151378c6fd5a10]) successfully deleted
I20260929 19:10:14.230582 11261 catalog_manager.cc:4841] TS 5528277035724bef87204f4738df1788 (127.9.236.2:36815): tablet 30051f9bf2d94b03b29470d4d287bdbd (table test-table [id=cb736d4107a54ca2b8151378c6fd5a10]) successfully deleted
I20260929 19:10:14.230873 11783 tablet_bootstrap.cc:492] T ba398039d4024d7ca3bf910af8e86172 P c52bd8c1c28746ecad9368f6316baf25: No bootstrap required, opened a new log
I20260929 19:10:14.230927 11783 ts_tablet_manager.cc:1397] T ba398039d4024d7ca3bf910af8e86172 P c52bd8c1c28746ecad9368f6316baf25: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260929 19:10:14.231052 11783 raft_consensus.cc:348] T ba398039d4024d7ca3bf910af8e86172 P c52bd8c1c28746ecad9368f6316baf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } }
I20260929 19:10:14.231117 11783 raft_consensus.cc:374] T ba398039d4024d7ca3bf910af8e86172 P c52bd8c1c28746ecad9368f6316baf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:14.231129 11783 raft_consensus.cc:729] T ba398039d4024d7ca3bf910af8e86172 P c52bd8c1c28746ecad9368f6316baf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c52bd8c1c28746ecad9368f6316baf25, State: Initialized, Role: FOLLOWER
I20260929 19:10:14.231190 11783 consensus_queue.cc:260] T ba398039d4024d7ca3bf910af8e86172 P c52bd8c1c28746ecad9368f6316baf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } }
I20260929 19:10:14.231256 11783 ts_tablet_manager.cc:1428] T ba398039d4024d7ca3bf910af8e86172 P c52bd8c1c28746ecad9368f6316baf25: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260929 19:10:14.247596 11481 tablet.cc:2369] T ba398039d4024d7ca3bf910af8e86172 P c52bd8c1c28746ecad9368f6316baf25: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:10:14.289773 11791 raft_consensus.cc:482] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:10:14.289968 11791 raft_consensus.cc:504] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } }
I20260929 19:10:14.290371 11791 leader_election.cc:288] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c52bd8c1c28746ecad9368f6316baf25 (127.9.236.1:33039), 5528277035724bef87204f4738df1788 (127.9.236.2:36815)
I20260929 19:10:14.294251 11781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.290849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:51397 (local address 127.9.236.1:33039)
0929 19:10:14.290929 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:14.290931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:14.290942 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:10:14.290977 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:14.290978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:14.290990 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:14.291057 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:14.291269 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.292058 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.293494 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.293911 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.293975 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.293990 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:14.294137 (+ 147us) server_negotiation.cc:299] Negotiation successful
0929 19:10:14.294179 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 19:10:14.294267 11653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.290607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:33039 (local address 127.9.236.3:51397)
0929 19:10:14.290688 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:14.290700 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:14.290753 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:14.291071 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:14.291072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:14.291074 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:14.291172 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.291174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.292150 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.292151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.293356 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.293358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.293980 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.293981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.294078 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.294082 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:10:14.294128 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 19:10:14.294171 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":3}
I20260929 19:10:14.294600 11782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.290782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:43031 (local address 127.9.236.2:36815)
0929 19:10:14.290842 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:14.290844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:14.290856 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:10:14.291020 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:14.291022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:14.291037 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:14.291105 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:14.291404 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.291933 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.293456 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.294098 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.294194 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.294208 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:14.294483 (+ 275us) server_negotiation.cc:299] Negotiation successful
0929 19:10:14.294524 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 19:10:14.294817 11434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba398039d4024d7ca3bf910af8e86172" candidate_uuid: "6783f18b0a934955aa840df9dbfc2a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c52bd8c1c28746ecad9368f6316baf25" is_pre_election: true
I20260929 19:10:14.294891 11813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.290628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:36815 (local address 127.9.236.3:43031)
0929 19:10:14.290780 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:14.290793 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:14.290860 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:14.291129 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:14.291130 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:14.291132 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:14.291209 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.291210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.292038 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.292039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.293357 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.293358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.294280 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.294283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.294380 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.294386 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 19:10:14.294722 (+ 336us) client_negotiation.cc:240] Negotiation successful
0929 19:10:14.294792 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":6,"thread_start_us":74,"threads_started":1}
I20260929 19:10:14.294917 11434 raft_consensus.cc:2454] T ba398039d4024d7ca3bf910af8e86172 P c52bd8c1c28746ecad9368f6316baf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6783f18b0a934955aa840df9dbfc2a1c in term 0.
I20260929 19:10:14.295207 11648 leader_election.cc:302] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6783f18b0a934955aa840df9dbfc2a1c, c52bd8c1c28746ecad9368f6316baf25; no voters:
I20260929 19:10:14.295207 11574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba398039d4024d7ca3bf910af8e86172" candidate_uuid: "6783f18b0a934955aa840df9dbfc2a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5528277035724bef87204f4738df1788" is_pre_election: true
I20260929 19:10:14.295336 11574 raft_consensus.cc:2454] T ba398039d4024d7ca3bf910af8e86172 P 5528277035724bef87204f4738df1788 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6783f18b0a934955aa840df9dbfc2a1c in term 0.
I20260929 19:10:14.295441 11791 raft_consensus.cc:2781] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:10:14.295488 11791 raft_consensus.cc:482] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:10:14.295508 11791 raft_consensus.cc:3037] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:14.296324 11791 raft_consensus.cc:504] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } }
I20260929 19:10:14.296486 11791 leader_election.cc:288] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c [CANDIDATE]: Term 1 election: Requested vote from peers c52bd8c1c28746ecad9368f6316baf25 (127.9.236.1:33039), 5528277035724bef87204f4738df1788 (127.9.236.2:36815)
I20260929 19:10:14.296662 11574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba398039d4024d7ca3bf910af8e86172" candidate_uuid: "6783f18b0a934955aa840df9dbfc2a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5528277035724bef87204f4738df1788"
I20260929 19:10:14.296729 11574 raft_consensus.cc:3037] T ba398039d4024d7ca3bf910af8e86172 P 5528277035724bef87204f4738df1788 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:14.297359 11434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba398039d4024d7ca3bf910af8e86172" candidate_uuid: "6783f18b0a934955aa840df9dbfc2a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c52bd8c1c28746ecad9368f6316baf25"
I20260929 19:10:14.297451 11434 raft_consensus.cc:3037] T ba398039d4024d7ca3bf910af8e86172 P c52bd8c1c28746ecad9368f6316baf25 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:14.297703 11574 raft_consensus.cc:2454] T ba398039d4024d7ca3bf910af8e86172 P 5528277035724bef87204f4738df1788 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6783f18b0a934955aa840df9dbfc2a1c in term 1.
I20260929 19:10:14.297896 11646 leader_election.cc:302] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5528277035724bef87204f4738df1788, 6783f18b0a934955aa840df9dbfc2a1c; no voters:
I20260929 19:10:14.297981 11791 raft_consensus.cc:2781] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:14.298146 11791 raft_consensus.cc:686] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c [term 1 LEADER]: Becoming Leader. State: Replica: 6783f18b0a934955aa840df9dbfc2a1c, State: Running, Role: LEADER
I20260929 19:10:14.298219 11791 consensus_queue.cc:237] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } }
I20260929 19:10:14.298306 11434 raft_consensus.cc:2454] T ba398039d4024d7ca3bf910af8e86172 P c52bd8c1c28746ecad9368f6316baf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6783f18b0a934955aa840df9dbfc2a1c in term 1.
I20260929 19:10:14.299059 11275 catalog_manager.cc:5495] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6783f18b0a934955aa840df9dbfc2a1c (127.9.236.3). New cstate: current_term: 1 leader_uuid: "6783f18b0a934955aa840df9dbfc2a1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } health_report { overall_health: HEALTHY } } }
W20260929 19:10:14.301159 11761 tablet.cc:2369] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 29 19:10:14 dist-test-slave-wftq krb5kdc[11231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709013, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:14.341477 11821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.333818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:41803 (local address 127.0.0.1:40892)
0929 19:10:14.334032 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:14.334047 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:14.334172 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:14.334345 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:14.334347 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:14.334829 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:14.334911 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.334913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.335560 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.335560 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.336350 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.336352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.336723 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.336724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.336780 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.337439 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:14.337445 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:14.339085 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:14.340482 (+ 1397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:14.340489 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:14.340491 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:14.340682 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:14.340850 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:14.340850 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:14.340851 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:14.340900 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:14.341093 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:14.341127 (+ 34us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:14.341183 (+ 56us) client_negotiation.cc:769] Sending connection context
0929 19:10:14.341242 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 19:10:14.341340 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":77,"threads_started":1}
I20260929 19:10:14.341485 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.333936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40892 (local address 127.9.236.62:41803)
0929 19:10:14.333977 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:14.333980 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:14.334190 (+ 210us) server_negotiation.cc:407] Connection header received
0929 19:10:14.334230 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:14.334231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:14.334241 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:14.334308 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:14.334995 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.335481 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.336442 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.336645 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.336713 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.339169 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:14.339170 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:14.339171 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:14.339178 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:14.340410 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:14.340745 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:14.340746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:14.340747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:14.340797 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:14.340946 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:14.340947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:14.340947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:14.341044 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:14.341092 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:14.341273 (+ 181us) server_negotiation.cc:299] Negotiation successful
0929 19:10:14.341387 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":17}
I20260929 19:10:14.343606 11275 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:40892:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 19:10:14.343824 11275 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:10:14.345435 11414 tablet_service.cc:1467] Processing CreateTablet for tablet d561b33726084bb69c6ed222d8a773ee (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3309015991dc4118b2831592a3d1ab4a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 19:10:14.345512 11554 tablet_service.cc:1467] Processing CreateTablet for tablet d561b33726084bb69c6ed222d8a773ee (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3309015991dc4118b2831592a3d1ab4a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 19:10:14.345588 11414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d561b33726084bb69c6ed222d8a773ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:14.345670 11554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d561b33726084bb69c6ed222d8a773ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:14.345626 11694 tablet_service.cc:1467] Processing CreateTablet for tablet d561b33726084bb69c6ed222d8a773ee (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3309015991dc4118b2831592a3d1ab4a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 19:10:14.345770 11694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d561b33726084bb69c6ed222d8a773ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:14.346974 11785 tablet_bootstrap.cc:492] T d561b33726084bb69c6ed222d8a773ee P 5528277035724bef87204f4738df1788: Bootstrap starting.
I20260929 19:10:14.347235 11787 tablet_bootstrap.cc:492] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c: Bootstrap starting.
I20260929 19:10:14.347251 11783 tablet_bootstrap.cc:492] T d561b33726084bb69c6ed222d8a773ee P c52bd8c1c28746ecad9368f6316baf25: Bootstrap starting.
I20260929 19:10:14.347337 11785 tablet_bootstrap.cc:654] T d561b33726084bb69c6ed222d8a773ee P 5528277035724bef87204f4738df1788: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:14.347695 11787 tablet_bootstrap.cc:654] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:14.347730 11783 tablet_bootstrap.cc:654] T d561b33726084bb69c6ed222d8a773ee P c52bd8c1c28746ecad9368f6316baf25: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:14.348065 11785 tablet_bootstrap.cc:492] T d561b33726084bb69c6ed222d8a773ee P 5528277035724bef87204f4738df1788: No bootstrap required, opened a new log
I20260929 19:10:14.348115 11785 ts_tablet_manager.cc:1397] T d561b33726084bb69c6ed222d8a773ee P 5528277035724bef87204f4738df1788: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:14.348232 11785 raft_consensus.cc:348] T d561b33726084bb69c6ed222d8a773ee P 5528277035724bef87204f4738df1788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } }
I20260929 19:10:14.348312 11785 raft_consensus.cc:374] T d561b33726084bb69c6ed222d8a773ee P 5528277035724bef87204f4738df1788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:14.348332 11785 raft_consensus.cc:729] T d561b33726084bb69c6ed222d8a773ee P 5528277035724bef87204f4738df1788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5528277035724bef87204f4738df1788, State: Initialized, Role: FOLLOWER
I20260929 19:10:14.348407 11783 tablet_bootstrap.cc:492] T d561b33726084bb69c6ed222d8a773ee P c52bd8c1c28746ecad9368f6316baf25: No bootstrap required, opened a new log
I20260929 19:10:14.348380 11785 consensus_queue.cc:260] T d561b33726084bb69c6ed222d8a773ee P 5528277035724bef87204f4738df1788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } }
I20260929 19:10:14.348444 11783 ts_tablet_manager.cc:1397] T d561b33726084bb69c6ed222d8a773ee P c52bd8c1c28746ecad9368f6316baf25: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:14.348489 11785 ts_tablet_manager.cc:1428] T d561b33726084bb69c6ed222d8a773ee P 5528277035724bef87204f4738df1788: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:14.348608 11787 tablet_bootstrap.cc:492] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c: No bootstrap required, opened a new log
I20260929 19:10:14.348536 11804 tablet_replica.cc:439] T d561b33726084bb69c6ed222d8a773ee P 5528277035724bef87204f4738df1788: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } } }
I20260929 19:10:14.348595 11783 raft_consensus.cc:348] T d561b33726084bb69c6ed222d8a773ee P c52bd8c1c28746ecad9368f6316baf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } }
I20260929 19:10:14.348659 11783 raft_consensus.cc:374] T d561b33726084bb69c6ed222d8a773ee P c52bd8c1c28746ecad9368f6316baf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:14.348646 11787 ts_tablet_manager.cc:1397] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:14.348654 11804 tablet_replica.cc:442] T d561b33726084bb69c6ed222d8a773ee P 5528277035724bef87204f4738df1788: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:10:14.348676 11783 raft_consensus.cc:729] T d561b33726084bb69c6ed222d8a773ee P c52bd8c1c28746ecad9368f6316baf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c52bd8c1c28746ecad9368f6316baf25, State: Initialized, Role: FOLLOWER
I20260929 19:10:14.348711 11783 consensus_queue.cc:260] T d561b33726084bb69c6ed222d8a773ee P c52bd8c1c28746ecad9368f6316baf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } }
I20260929 19:10:14.348791 11783 ts_tablet_manager.cc:1428] T d561b33726084bb69c6ed222d8a773ee P c52bd8c1c28746ecad9368f6316baf25: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:14.348845 11788 tablet_replica.cc:439] T d561b33726084bb69c6ed222d8a773ee P c52bd8c1c28746ecad9368f6316baf25: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } } }
I20260929 19:10:14.348934 11788 tablet_replica.cc:442] T d561b33726084bb69c6ed222d8a773ee P c52bd8c1c28746ecad9368f6316baf25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:10:14.348918 11787 raft_consensus.cc:348] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } }
I20260929 19:10:14.348986 11787 raft_consensus.cc:374] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:14.349007 11787 raft_consensus.cc:729] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6783f18b0a934955aa840df9dbfc2a1c, State: Initialized, Role: FOLLOWER
I20260929 19:10:14.349050 11787 consensus_queue.cc:260] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } }
I20260929 19:10:14.349159 11787 ts_tablet_manager.cc:1428] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260929 19:10:14.349231 11814 tablet_replica.cc:439] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } } }
I20260929 19:10:14.349291 11814 tablet_replica.cc:442] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:10:14.442220 11814 raft_consensus.cc:482] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:10:14.442327 11814 raft_consensus.cc:504] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } }
I20260929 19:10:14.442509 11814 leader_election.cc:288] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c52bd8c1c28746ecad9368f6316baf25 (127.9.236.1:33039), 5528277035724bef87204f4738df1788 (127.9.236.2:36815)
I20260929 19:10:14.442816 11434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d561b33726084bb69c6ed222d8a773ee" candidate_uuid: "6783f18b0a934955aa840df9dbfc2a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c52bd8c1c28746ecad9368f6316baf25" is_pre_election: true
I20260929 19:10:14.442799 11574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d561b33726084bb69c6ed222d8a773ee" candidate_uuid: "6783f18b0a934955aa840df9dbfc2a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5528277035724bef87204f4738df1788" is_pre_election: true
I20260929 19:10:14.442968 11434 raft_consensus.cc:2454] T d561b33726084bb69c6ed222d8a773ee P c52bd8c1c28746ecad9368f6316baf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6783f18b0a934955aa840df9dbfc2a1c in term 0.
I20260929 19:10:14.442991 11574 raft_consensus.cc:2454] T d561b33726084bb69c6ed222d8a773ee P 5528277035724bef87204f4738df1788 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6783f18b0a934955aa840df9dbfc2a1c in term 0.
I20260929 19:10:14.443156 11646 leader_election.cc:302] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5528277035724bef87204f4738df1788, 6783f18b0a934955aa840df9dbfc2a1c; no voters:
I20260929 19:10:14.443276 11814 raft_consensus.cc:2781] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:10:14.443320 11814 raft_consensus.cc:482] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:10:14.443336 11814 raft_consensus.cc:3037] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:14.444209 11814 raft_consensus.cc:504] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } }
I20260929 19:10:14.444341 11814 leader_election.cc:288] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c [CANDIDATE]: Term 1 election: Requested vote from peers c52bd8c1c28746ecad9368f6316baf25 (127.9.236.1:33039), 5528277035724bef87204f4738df1788 (127.9.236.2:36815)
I20260929 19:10:14.444525 11574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d561b33726084bb69c6ed222d8a773ee" candidate_uuid: "6783f18b0a934955aa840df9dbfc2a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5528277035724bef87204f4738df1788"
I20260929 19:10:14.444595 11574 raft_consensus.cc:3037] T d561b33726084bb69c6ed222d8a773ee P 5528277035724bef87204f4738df1788 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:14.444540 11434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d561b33726084bb69c6ed222d8a773ee" candidate_uuid: "6783f18b0a934955aa840df9dbfc2a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c52bd8c1c28746ecad9368f6316baf25"
I20260929 19:10:14.444640 11434 raft_consensus.cc:3037] T d561b33726084bb69c6ed222d8a773ee P c52bd8c1c28746ecad9368f6316baf25 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:14.445461 11434 raft_consensus.cc:2454] T d561b33726084bb69c6ed222d8a773ee P c52bd8c1c28746ecad9368f6316baf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6783f18b0a934955aa840df9dbfc2a1c in term 1.
I20260929 19:10:14.445596 11574 raft_consensus.cc:2454] T d561b33726084bb69c6ed222d8a773ee P 5528277035724bef87204f4738df1788 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6783f18b0a934955aa840df9dbfc2a1c in term 1.
I20260929 19:10:14.445715 11648 leader_election.cc:302] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6783f18b0a934955aa840df9dbfc2a1c, c52bd8c1c28746ecad9368f6316baf25; no voters:
I20260929 19:10:14.445845 11814 raft_consensus.cc:2781] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:14.445926 11814 raft_consensus.cc:686] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c [term 1 LEADER]: Becoming Leader. State: Replica: 6783f18b0a934955aa840df9dbfc2a1c, State: Running, Role: LEADER
I20260929 19:10:14.445986 11814 consensus_queue.cc:237] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } }
I20260929 19:10:14.446352 11791 tablet_replica.cc:439] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c: TxnStatusTablet state changed. Reason: New leader 6783f18b0a934955aa840df9dbfc2a1c. Latest consensus state: current_term: 1 leader_uuid: "6783f18b0a934955aa840df9dbfc2a1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } } }
I20260929 19:10:14.446449 11791 tablet_replica.cc:442] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c: This TxnStatusTablet replica's current role is: LEADER
I20260929 19:10:14.446660 11826 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 19:10:14.446786 11826 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 19:10:14.446712 11275 catalog_manager.cc:5495] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6783f18b0a934955aa840df9dbfc2a1c (127.9.236.3). New cstate: current_term: 1 leader_uuid: "6783f18b0a934955aa840df9dbfc2a1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } health_report { overall_health: HEALTHY } } }
I20260929 19:10:14.467872 11821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.465148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:45121 (local address 127.0.0.1:49414)
0929 19:10:14.465207 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:14.465221 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:14.465277 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:14.465496 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:14.465497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:14.465499 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:10:14.465565 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.465569 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.466270 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.466271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.466886 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.466957 (+ 71us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.467338 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.467338 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.467410 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.467426 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:10:14.467676 (+ 250us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:10:14.467685 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 19:10:14.467728 (+ 43us) client_negotiation.cc:240] Negotiation successful
0929 19:10:14.467779 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260929 19:10:14.467844 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.465225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49414 (local address 127.9.236.3:45121)
0929 19:10:14.465291 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:14.465293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:14.465305 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:10:14.465392 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:14.465393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:14.465417 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:14.465485 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:14.465630 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.466182 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.467055 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.467278 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.467336 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.467507 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:14.467623 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:14.467685 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:14.467732 (+ 47us) server_negotiation.cc:299] Negotiation successful
0929 19:10:14.467756 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 19:10:14.469385 11434 raft_consensus.cc:1261] T d561b33726084bb69c6ed222d8a773ee P c52bd8c1c28746ecad9368f6316baf25 [term 1 FOLLOWER]: Refusing update from remote peer 6783f18b0a934955aa840df9dbfc2a1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:14.469406 11574 raft_consensus.cc:1261] T d561b33726084bb69c6ed222d8a773ee P 5528277035724bef87204f4738df1788 [term 1 FOLLOWER]: Refusing update from remote peer 6783f18b0a934955aa840df9dbfc2a1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:14.469738 11814 consensus_queue.cc:1035] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:14.469921 11791 consensus_queue.cc:1035] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:14.470628 11788 tablet_replica.cc:439] T d561b33726084bb69c6ed222d8a773ee P c52bd8c1c28746ecad9368f6316baf25: TxnStatusTablet state changed. Reason: New leader 6783f18b0a934955aa840df9dbfc2a1c. Latest consensus state: current_term: 1 leader_uuid: "6783f18b0a934955aa840df9dbfc2a1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } } }
I20260929 19:10:14.470724 11788 tablet_replica.cc:442] T d561b33726084bb69c6ed222d8a773ee P c52bd8c1c28746ecad9368f6316baf25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:10:14.470800 11804 tablet_replica.cc:439] T d561b33726084bb69c6ed222d8a773ee P 5528277035724bef87204f4738df1788: TxnStatusTablet state changed. Reason: New leader 6783f18b0a934955aa840df9dbfc2a1c. Latest consensus state: current_term: 1 leader_uuid: "6783f18b0a934955aa840df9dbfc2a1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } } }
I20260929 19:10:14.470875 11804 tablet_replica.cc:442] T d561b33726084bb69c6ed222d8a773ee P 5528277035724bef87204f4738df1788: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:10:14.471524 11814 tablet_replica.cc:439] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6783f18b0a934955aa840df9dbfc2a1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } } }
I20260929 19:10:14.471622 11814 tablet_replica.cc:442] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c: This TxnStatusTablet replica's current role is: LEADER
I20260929 19:10:14.472111 11788 tablet_replica.cc:439] T d561b33726084bb69c6ed222d8a773ee P c52bd8c1c28746ecad9368f6316baf25: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6783f18b0a934955aa840df9dbfc2a1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } } }
I20260929 19:10:14.472190 11788 tablet_replica.cc:442] T d561b33726084bb69c6ed222d8a773ee P c52bd8c1c28746ecad9368f6316baf25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:10:14.472843 11791 tablet_replica.cc:439] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6783f18b0a934955aa840df9dbfc2a1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } } }
I20260929 19:10:14.472924 11791 tablet_replica.cc:442] T d561b33726084bb69c6ed222d8a773ee P 6783f18b0a934955aa840df9dbfc2a1c: This TxnStatusTablet replica's current role is: LEADER
I20260929 19:10:14.474090 11804 tablet_replica.cc:439] T d561b33726084bb69c6ed222d8a773ee P 5528277035724bef87204f4738df1788: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6783f18b0a934955aa840df9dbfc2a1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 } } peers { permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 } } peers { permanent_uuid: "6783f18b0a934955aa840df9dbfc2a1c" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 45121 } } }
I20260929 19:10:14.474193 11804 tablet_replica.cc:442] T d561b33726084bb69c6ed222d8a773ee P 5528277035724bef87204f4738df1788: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:10:14.476212 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.473890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49422 (local address 127.9.236.3:45121)
0929 19:10:14.473950 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:14.473952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:14.473962 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:10:14.473982 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:14.473983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:14.473995 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:14.474051 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:14.474209 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.474846 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.475619 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.475813 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.475863 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.475982 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:14.476015 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:14.476061 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:14.476123 (+ 62us) server_negotiation.cc:299] Negotiation successful
0929 19:10:14.476141 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 19:10:14.481513 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.478946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:52435 (local address 127.9.236.3:45121)
0929 19:10:14.479001 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:14.479002 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:14.479009 (+ 7us) server_negotiation.cc:407] Connection header received
0929 19:10:14.479026 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:14.479026 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:14.479033 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:14.479081 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:14.479243 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.479675 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.480829 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.481004 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.481060 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.481276 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:14.481301 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:14.481338 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:14.481423 (+ 85us) server_negotiation.cc:299] Negotiation successful
0929 19:10:14.481438 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 19:10:14.481631 11766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.478751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:45121 (local address 127.9.236.3:52435)
0929 19:10:14.478835 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:14.478846 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:14.478882 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:14.479104 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:14.479105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:14.479106 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:10:14.479179 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.479182 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.479744 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.479745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.480741 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:14.480744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.481077 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.481078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:14.481177 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.481195 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:10:14.481358 (+ 163us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:10:14.481359 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 19:10:14.481419 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 19:10:14.481464 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":1}
I20260929 19:10:14.483852 11434 raft_consensus.cc:1261] T ba398039d4024d7ca3bf910af8e86172 P c52bd8c1c28746ecad9368f6316baf25 [term 1 FOLLOWER]: Refusing update from remote peer 6783f18b0a934955aa840df9dbfc2a1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:14.483932 11574 raft_consensus.cc:1261] T ba398039d4024d7ca3bf910af8e86172 P 5528277035724bef87204f4738df1788 [term 1 FOLLOWER]: Refusing update from remote peer 6783f18b0a934955aa840df9dbfc2a1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:14.484066 11831 consensus_queue.cc:1035] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c52bd8c1c28746ecad9368f6316baf25" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:14.484169 11831 consensus_queue.cc:1035] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5528277035724bef87204f4738df1788" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:14.496714 11276 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:14.496831 11276 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:14.497514 11276 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 760cd1a69d5d44e5ab5e287c0e286071: Sending DeleteTablet for 3 replicas of tablet ba398039d4024d7ca3bf910af8e86172
I20260929 19:10:14.497833 11554 tablet_service.cc:1514] Processing DeleteTablet for tablet ba398039d4024d7ca3bf910af8e86172 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:14 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:38850
I20260929 19:10:14.497874 11694 tablet_service.cc:1514] Processing DeleteTablet for tablet ba398039d4024d7ca3bf910af8e86172 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:14 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:49400
I20260929 19:10:14.497939 11414 tablet_service.cc:1514] Processing DeleteTablet for tablet ba398039d4024d7ca3bf910af8e86172 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:14 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:36634
I20260929 19:10:14.497987 11809 tablet_replica.cc:330] T ba398039d4024d7ca3bf910af8e86172 P 5528277035724bef87204f4738df1788: stopping tablet replica
I20260929 19:10:14.498054 11809 raft_consensus.cc:2229] T ba398039d4024d7ca3bf910af8e86172 P 5528277035724bef87204f4738df1788 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:14.498068 11808 tablet_replica.cc:330] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c: stopping tablet replica
I20260929 19:10:14.498101 11809 raft_consensus.cc:2258] T ba398039d4024d7ca3bf910af8e86172 P 5528277035724bef87204f4738df1788 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:14.498109 11808 raft_consensus.cc:2229] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:10:14.498206 11808 raft_consensus.cc:2258] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:14.498288 11807 tablet_replica.cc:330] T ba398039d4024d7ca3bf910af8e86172 P c52bd8c1c28746ecad9368f6316baf25: stopping tablet replica
I20260929 19:10:14.498356 11807 raft_consensus.cc:2229] T ba398039d4024d7ca3bf910af8e86172 P c52bd8c1c28746ecad9368f6316baf25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:14.498402 11807 raft_consensus.cc:2258] T ba398039d4024d7ca3bf910af8e86172 P c52bd8c1c28746ecad9368f6316baf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:14.498445 11809 ts_tablet_manager.cc:1905] T ba398039d4024d7ca3bf910af8e86172 P 5528277035724bef87204f4738df1788: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:14.498746 11808 ts_tablet_manager.cc:1905] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:14.500213 11808 ts_tablet_manager.cc:1918] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:10:14.500272 11808 log.cc:1196] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/ba398039d4024d7ca3bf910af8e86172
I20260929 19:10:14.500270 11809 ts_tablet_manager.cc:1918] T ba398039d4024d7ca3bf910af8e86172 P 5528277035724bef87204f4738df1788: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:10:14.500325 11809 log.cc:1196] T ba398039d4024d7ca3bf910af8e86172 P 5528277035724bef87204f4738df1788: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/ba398039d4024d7ca3bf910af8e86172
I20260929 19:10:14.500326 11807 ts_tablet_manager.cc:1905] T ba398039d4024d7ca3bf910af8e86172 P c52bd8c1c28746ecad9368f6316baf25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:14.500589 11808 ts_tablet_manager.cc:1939] T ba398039d4024d7ca3bf910af8e86172 P 6783f18b0a934955aa840df9dbfc2a1c: Deleting consensus metadata
I20260929 19:10:14.500594 11809 ts_tablet_manager.cc:1939] T ba398039d4024d7ca3bf910af8e86172 P 5528277035724bef87204f4738df1788: Deleting consensus metadata
I20260929 19:10:14.500945 11261 catalog_manager.cc:4841] TS 6783f18b0a934955aa840df9dbfc2a1c (127.9.236.3:45121): tablet ba398039d4024d7ca3bf910af8e86172 (table test-table [id=b7fd6dbfd7bc421fada7e63006af9940]) successfully deleted
I20260929 19:10:14.501015 11261 catalog_manager.cc:4841] TS 5528277035724bef87204f4738df1788 (127.9.236.2:36815): tablet ba398039d4024d7ca3bf910af8e86172 (table test-table [id=b7fd6dbfd7bc421fada7e63006af9940]) successfully deleted
I20260929 19:10:14.502012 11807 ts_tablet_manager.cc:1918] T ba398039d4024d7ca3bf910af8e86172 P c52bd8c1c28746ecad9368f6316baf25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:10:14.502049 11807 log.cc:1196] T ba398039d4024d7ca3bf910af8e86172 P c52bd8c1c28746ecad9368f6316baf25: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790709008521303-10160-0/minicluster-data/ts-0/wal/wals/ba398039d4024d7ca3bf910af8e86172
I20260929 19:10:14.502188 11807 ts_tablet_manager.cc:1939] T ba398039d4024d7ca3bf910af8e86172 P c52bd8c1c28746ecad9368f6316baf25: Deleting consensus metadata
I20260929 19:10:14.502602 11263 catalog_manager.cc:4841] TS c52bd8c1c28746ecad9368f6316baf25 (127.9.236.1:33039): tablet ba398039d4024d7ca3bf910af8e86172 (table test-table [id=b7fd6dbfd7bc421fada7e63006af9940]) successfully deleted
Sep 29 19:10:14 dist-test-slave-wftq krb5kdc[11231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709014, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.236.1@KRBTEST.COM
I20260929 19:10:14.510329 11781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.500492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36650 (local address 127.9.236.1:33039)
0929 19:10:14.500870 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:14.500872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:14.501253 (+ 381us) server_negotiation.cc:407] Connection header received
0929 19:10:14.501347 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:14.501348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:14.501362 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:14.501389 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:14.502111 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.502894 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.503714 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.504032 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.504104 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.506952 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:14.506961 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:14.506962 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:14.506969 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:14.508863 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:14.509237 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:14.509241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:14.509242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:14.509311 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:14.509701 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:14.509702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:14.509702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:14.509886 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:14.510022 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:14.510149 (+ 127us) server_negotiation.cc:299] Negotiation successful
0929 19:10:14.510234 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353}
W20260929 19:10:14.510560 11474 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36650
I20260929 19:10:14.517582 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:14.511502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40902 (local address 127.9.236.62:41803)
0929 19:10:14.511550 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:14.511552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:14.511562 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:10:14.511638 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:14.511639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:14.511649 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:14.511714 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:14.512197 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.512912 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.513645 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:14.513925 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:14.514001 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:14.515074 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:14.515076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:14.515076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:14.515083 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:14.516513 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:14.516823 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:14.516825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:14.516825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:14.516892 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:14.517063 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:14.517064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:14.517064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:14.517211 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:14.517309 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:14.517396 (+ 87us) server_negotiation.cc:299] Negotiation successful
0929 19:10:14.517485 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260929 19:10:14.517779 11276 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40902
I20260929 19:10:14.518543 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 11349
I20260929 19:10:14.526541 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 11489
I20260929 19:10:14.533594 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 11629
I20260929 19:10:14.539508 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 11244
2026-09-29T19:10:14Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2232 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:10:14 dist-test-slave-wftq krb5kdc[11843](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:10:14 dist-test-slave-wftq krb5kdc[11843](info): set up 2 sockets
Sep 29 19:10:14 dist-test-slave-wftq krb5kdc[11843](info): commencing operation
krb5kdc: starting...
W20260929 19:10:15.500878 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:10:15 dist-test-slave-wftq krb5kdc[11843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:10:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:10:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:10:15.620188 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:40013
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:34675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:40013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:15.707432 11856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:15.707692 11856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:15.707717 11856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:15.709159 11856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:15.709209 11856 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 19:10:15.709223 11856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:15.709234 11856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:15.709249 11856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:15.709260 11856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:15.710942 11856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:34675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:40013
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:40013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:15.711201 11856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:15.711457 11856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:15.716681 11864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:15.716733 11861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:15.716820 11862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:15.716854 11856 server_base.cc:1034] running on GCE node
I20260929 19:10:15.717321 11856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:15.717664 11856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:15.718801 11856 hybrid_clock.cc:648] HybridClock initialized: now 1790709015718785 us; error 35 us; skew 500 ppm
Sep 29 19:10:15 dist-test-slave-wftq krb5kdc[11843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709015, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:15.721369 11856 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:10:15.721812 11856 webserver.cc:460] Webserver started at http://127.9.236.62:41109/ using document root <none> and password file <none>
I20260929 19:10:15.722018 11856 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:15.722061 11856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:15.722158 11856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:15.723207 11856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "492fa2d090474b6a83e5d270db635417"
format_stamp: "Formatted at 2026-09-29 19:10:15 on dist-test-slave-wftq"
server_key: "f94fea0b9bf7781050c7bb8633c2ca2c"
server_key_iv: "97192ffbf568bab7e8b6119332f7523a"
server_key_version: "encryptionkey@0"
I20260929 19:10:15.723510 11856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "492fa2d090474b6a83e5d270db635417"
format_stamp: "Formatted at 2026-09-29 19:10:15 on dist-test-slave-wftq"
server_key: "f94fea0b9bf7781050c7bb8633c2ca2c"
server_key_iv: "97192ffbf568bab7e8b6119332f7523a"
server_key_version: "encryptionkey@0"
I20260929 19:10:15.724805 11856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:10:15.725611 11871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:15.725821 11856 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 19:10:15.725912 11856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "492fa2d090474b6a83e5d270db635417"
format_stamp: "Formatted at 2026-09-29 19:10:15 on dist-test-slave-wftq"
server_key: "f94fea0b9bf7781050c7bb8633c2ca2c"
server_key_iv: "97192ffbf568bab7e8b6119332f7523a"
server_key_version: "encryptionkey@0"
I20260929 19:10:15.725977 11856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:15.741458 11856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:15.743458 11856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:15.743599 11856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:15.748425 11856 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:40013
I20260929 19:10:15.748550 11933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:40013 every 8 connection(s)
I20260929 19:10:15.748821 11856 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:10:15.749617 11934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:15.752285 11934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417: Bootstrap starting.
I20260929 19:10:15.752926 11934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:15.753257 11934 log.cc:824] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:15.754135 11934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417: No bootstrap required, opened a new log
I20260929 19:10:15.755154 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 11856
I20260929 19:10:15.755245 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:10:15.755398 10160 external_mini_cluster.cc:1428] Setting key d365c021b1dd523a7aed91ac19e8e006
I20260929 19:10:15.756613 11934 raft_consensus.cc:348] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492fa2d090474b6a83e5d270db635417" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40013 } }
I20260929 19:10:15.756776 11934 raft_consensus.cc:374] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:15.756810 11934 raft_consensus.cc:729] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 492fa2d090474b6a83e5d270db635417, State: Initialized, Role: FOLLOWER
I20260929 19:10:15.757017 11934 consensus_queue.cc:260] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492fa2d090474b6a83e5d270db635417" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40013 } }
I20260929 19:10:15.757135 11934 raft_consensus.cc:388] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:10:15.757169 11934 raft_consensus.cc:482] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:10:15.757220 11934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:15.757897 11934 raft_consensus.cc:504] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492fa2d090474b6a83e5d270db635417" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40013 } }
I20260929 19:10:15.757990 11934 leader_election.cc:302] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 492fa2d090474b6a83e5d270db635417; no voters:
I20260929 19:10:15.758168 11934 leader_election.cc:288] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:10:15.758241 11939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:15.758433 11934 sys_catalog.cc:564] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:15.758534 11939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417 [term 1 LEADER]: Becoming Leader. State: Replica: 492fa2d090474b6a83e5d270db635417, State: Running, Role: LEADER
I20260929 19:10:15.758690 11939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492fa2d090474b6a83e5d270db635417" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40013 } }
I20260929 19:10:15.759127 11940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "492fa2d090474b6a83e5d270db635417" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492fa2d090474b6a83e5d270db635417" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40013 } } }
I20260929 19:10:15.759227 11940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:15.759406 11941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 492fa2d090474b6a83e5d270db635417. Latest consensus state: current_term: 1 leader_uuid: "492fa2d090474b6a83e5d270db635417" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492fa2d090474b6a83e5d270db635417" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40013 } } }
I20260929 19:10:15.759459 11941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417 [sys.catalog]: This master's current role is: LEADER
W20260929 19:10:15.760398 11955 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:10:15.760456 11955 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:10:15.760502 11948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:15.760643 11948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:10:15.762353 11948 catalog_manager.cc:1347] Generated new cluster ID: 8ef36420ca534e63b2dbc9f52f645bdc
I20260929 19:10:15.762419 11948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:10:15 dist-test-slave-wftq krb5kdc[11843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:15.766074 11938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:15.755991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34292 (local address 127.9.236.62:40013)
0929 19:10:15.756193 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:15.756201 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:15.756230 (+ 29us) server_negotiation.cc:407] Connection header received
0929 19:10:15.756591 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:15.756598 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:15.756659 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:15.756796 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:15.757669 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:15.758890 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:15.760023 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:15.760536 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:15.760638 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:15.763112 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:15.763117 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:15.763118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:15.763130 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:15.764621 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:15.764919 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:15.764922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:15.764923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:15.765020 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:15.765238 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:15.765238 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:15.765239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:15.765476 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:15.765633 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:15.765734 (+ 101us) server_negotiation.cc:299] Negotiation successful
0929 19:10:15.765874 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":95,"threads_started":1}
I20260929 19:10:15.786747 11948 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:10:15.787336 11948 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:15.789994 11948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417: Generated new TSK 0
I20260929 19:10:15.790236 11948 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.236.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
I20260929 19:10:15.819720 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:40013
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:34675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:15.902529 11960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:15.902729 11960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:15.902755 11960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:15.903873 11960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:15.903914 11960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:15.903934 11960 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 19:10:15.903944 11960 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 19:10:15.903961 11960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:10:15.905383 11960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:34675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.236.62:40013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:15.905582 11960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:15.905793 11960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:10:15.910784 11960 server_base.cc:1034] running on GCE node
W20260929 19:10:15.910724 11965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:15.910722 11966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:15.910727 11968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:15.911238 11960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:15.911428 11960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:15.912560 11960 hybrid_clock.cc:648] HybridClock initialized: now 1790709015912541 us; error 35 us; skew 500 ppm
Sep 29 19:10:15 dist-test-slave-wftq krb5kdc[11843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709015, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:15.915016 11960 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:10:15.915441 11960 webserver.cc:460] Webserver started at http://127.9.236.1:42605/ using document root <none> and password file <none>
I20260929 19:10:15.915629 11960 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:15.915679 11960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:15.915795 11960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:15.916908 11960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "6500e0146cf84a229ff00c3d1e1c7f51"
format_stamp: "Formatted at 2026-09-29 19:10:15 on dist-test-slave-wftq"
server_key: "e3d00c9cfd5a3310311f43cf7731d075"
server_key_iv: "31da8a8ffbd5f5e8112aac3c9ea08c4e"
server_key_version: "encryptionkey@0"
I20260929 19:10:15.917272 11960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "6500e0146cf84a229ff00c3d1e1c7f51"
format_stamp: "Formatted at 2026-09-29 19:10:15 on dist-test-slave-wftq"
server_key: "e3d00c9cfd5a3310311f43cf7731d075"
server_key_iv: "31da8a8ffbd5f5e8112aac3c9ea08c4e"
server_key_version: "encryptionkey@0"
I20260929 19:10:15.918469 11960 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:15.919231 11975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:15.919442 11960 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:15.919523 11960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "6500e0146cf84a229ff00c3d1e1c7f51"
format_stamp: "Formatted at 2026-09-29 19:10:15 on dist-test-slave-wftq"
server_key: "e3d00c9cfd5a3310311f43cf7731d075"
server_key_iv: "31da8a8ffbd5f5e8112aac3c9ea08c4e"
server_key_version: "encryptionkey@0"
I20260929 19:10:15.919584 11960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:15.930490 11960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:15.932219 11960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:15.932345 11960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:15.932863 11960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:15.932906 11960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:15.932934 11960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:15.932950 11960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 19:10:15 dist-test-slave-wftq krb5kdc[11843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709015, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:15.940001 11960 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:33007
I20260929 19:10:15.940096 12090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:33007 every 8 connection(s)
I20260929 19:10:15.940378 11960 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:10:15.943989 11938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:15.933394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:48233 (local address 127.9.236.62:40013)
0929 19:10:15.933453 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:15.933455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:15.933699 (+ 244us) server_negotiation.cc:407] Connection header received
0929 19:10:15.934326 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:15.934328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:15.934341 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:15.934394 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:15.935086 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:15.935659 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:15.936739 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:15.936960 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:15.937017 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:15.940057 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:15.940059 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:15.940059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:15.940071 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:15.942701 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:15.943195 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:15.943195 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:15.943196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:15.943245 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:15.943431 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:15.943431 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:15.943432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:15.943549 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:15.943599 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:15.943839 (+ 240us) server_negotiation.cc:299] Negotiation successful
0929 19:10:15.943912 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 19:10:15.944178 11984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:15.933382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:40013 (local address 127.9.236.1:48233)
0929 19:10:15.933562 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:15.933604 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:15.934213 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:15.934484 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:15.934489 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:15.934770 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:15.934991 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:15.934996 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:15.935741 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:15.935743 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:15.936656 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:15.936659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:15.937044 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:15.937045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:15.937195 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:15.937734 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:15.937747 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:15.939954 (+ 2207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:15.942813 (+ 2859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:15.942820 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:15.942822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:15.943092 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:15.943288 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:15.943289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:15.943289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:15.943382 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:15.943604 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:15.943607 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:15.943736 (+ 129us) client_negotiation.cc:769] Sending connection context
0929 19:10:15.943820 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 19:10:15.943960 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":93,"threads_started":1}
I20260929 19:10:15.944206 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 11960
I20260929 19:10:15.944286 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:10:15.944437 10160 external_mini_cluster.cc:1428] Setting key c9fa26b6d770193a1b3569e55d1bfa5f
I20260929 19:10:15.944974 12091 heartbeater.cc:347] Connected to a master server at 127.9.236.62:40013
I20260929 19:10:15.945080 12091 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:15.945289 12091 heartbeater.cc:511] Master 127.9.236.62:40013 requested a full tablet report, sending...
I20260929 19:10:15.945776 11888 ts_manager.cc:194] Registered new tserver with Master: 6500e0146cf84a229ff00c3d1e1c7f51 (127.9.236.1:33007)
I20260929 19:10:15.946558 11888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:48233
I20260929 19:10:15.952057 11938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:15.946831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:48873 (local address 127.9.236.62:40013)
0929 19:10:15.946870 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:15.946872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:15.946880 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:10:15.946898 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:15.946899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:15.946908 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:15.946969 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:15.947366 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:15.947909 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:15.948613 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:15.948816 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:15.948874 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:15.949863 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:15.949864 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:15.949864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:15.949871 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:15.951086 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:15.951383 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:15.951384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:15.951384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:15.951434 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:15.951599 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:15.951600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:15.951600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:15.951704 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:15.951759 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:15.951886 (+ 127us) server_negotiation.cc:299] Negotiation successful
0929 19:10:15.951962 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:10:15.952067 12097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:15.945985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:40013 (local address 127.9.236.1:48873)
0929 19:10:15.946191 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:15.946204 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:15.946265 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:15.946992 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:15.946993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:15.947192 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:15.947273 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:15.947276 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:15.948010 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:15.948011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:15.948541 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:15.948543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:15.948936 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:15.948937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:15.949003 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:15.949428 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:15.949432 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:15.949759 (+ 327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:15.951162 (+ 1403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:15.951163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:15.951164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:15.951324 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:15.951485 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:15.951485 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:15.951486 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:15.951543 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:15.951756 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:15.951757 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:15.951800 (+ 43us) client_negotiation.cc:769] Sending connection context
0929 19:10:15.951859 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 19:10:15.951964 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":1,"thread_start_us":119,"threads_started":1}
WARNING: no policy specified for kudu/127.9.236.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
I20260929 19:10:15.971263 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:40013
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:34675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:16.056964 12100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:16.057226 12100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:16.057258 12100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:16.058461 12100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:16.058504 12100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:16.058530 12100 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 19:10:16.058550 12100 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 19:10:16.058575 12100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:10:16.060096 12100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:34675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.236.62:40013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:16.060350 12100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:16.060575 12100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:16.065817 12105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:16.065817 12108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:16.065871 12106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:16.066079 12100 server_base.cc:1034] running on GCE node
I20260929 19:10:16.066373 12100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:16.066627 12100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:16.067816 12100 hybrid_clock.cc:648] HybridClock initialized: now 1790709016067795 us; error 38 us; skew 500 ppm
Sep 29 19:10:16 dist-test-slave-wftq krb5kdc[11843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709016, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:16.070832 12100 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:10:16.071352 12100 webserver.cc:460] Webserver started at http://127.9.236.2:42485/ using document root <none> and password file <none>
I20260929 19:10:16.071555 12100 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:16.071602 12100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:16.071713 12100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:16.072906 12100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "433eed7b49a44911bd5764d0f5a97bcc"
format_stamp: "Formatted at 2026-09-29 19:10:16 on dist-test-slave-wftq"
server_key: "8bcadc058e6b7d0b6054ba1f60c1fd88"
server_key_iv: "c430a8054f2a19cad53adfe43eae10dc"
server_key_version: "encryptionkey@0"
I20260929 19:10:16.073292 12100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "433eed7b49a44911bd5764d0f5a97bcc"
format_stamp: "Formatted at 2026-09-29 19:10:16 on dist-test-slave-wftq"
server_key: "8bcadc058e6b7d0b6054ba1f60c1fd88"
server_key_iv: "c430a8054f2a19cad53adfe43eae10dc"
server_key_version: "encryptionkey@0"
I20260929 19:10:16.074779 12100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:16.075692 12115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:16.075944 12100 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:16.076038 12100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "433eed7b49a44911bd5764d0f5a97bcc"
format_stamp: "Formatted at 2026-09-29 19:10:16 on dist-test-slave-wftq"
server_key: "8bcadc058e6b7d0b6054ba1f60c1fd88"
server_key_iv: "c430a8054f2a19cad53adfe43eae10dc"
server_key_version: "encryptionkey@0"
I20260929 19:10:16.076126 12100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:16.090656 12100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:16.092833 12100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:16.092984 12100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:16.093613 12100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:16.093659 12100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:16.093691 12100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:16.093712 12100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 19:10:16 dist-test-slave-wftq krb5kdc[11843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709016, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:16.101785 12100 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:37111
I20260929 19:10:16.101912 12230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:37111 every 8 connection(s)
I20260929 19:10:16.102226 12100 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:10:16.104436 11938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.094259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:52271 (local address 127.9.236.62:40013)
0929 19:10:16.094317 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:16.094319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:16.094573 (+ 254us) server_negotiation.cc:407] Connection header received
0929 19:10:16.095161 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:16.095162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:16.095174 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:16.095226 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:16.096076 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.096766 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.098484 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.098680 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.098750 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.101777 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:16.101778 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:16.101779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:16.101788 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:16.103199 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:16.103563 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:16.103564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:16.103564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:16.103611 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:16.103847 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:16.103848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:16.103848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:16.103951 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:16.104006 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:16.104271 (+ 265us) server_negotiation.cc:299] Negotiation successful
0929 19:10:16.104343 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 19:10:16.104653 12124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.094217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:40013 (local address 127.9.236.2:52271)
0929 19:10:16.094427 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:16.094471 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:16.095046 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:16.095315 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:16.095319 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:16.095703 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:16.095994 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.095999 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.096889 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.096891 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.098369 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.098375 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.098777 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.098778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.098943 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.099600 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:16.099620 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:16.101676 (+ 2056us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:16.103275 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:16.103281 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:16.103284 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:16.103499 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:16.103693 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:16.103694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:16.103694 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:16.103791 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:16.104004 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:16.104009 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:16.104156 (+ 147us) client_negotiation.cc:769] Sending connection context
0929 19:10:16.104246 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 19:10:16.104432 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":100,"threads_started":1}
I20260929 19:10:16.105242 12231 heartbeater.cc:347] Connected to a master server at 127.9.236.62:40013
I20260929 19:10:16.105331 12231 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:16.105525 12231 heartbeater.cc:511] Master 127.9.236.62:40013 requested a full tablet report, sending...
I20260929 19:10:16.105937 11888 ts_manager.cc:194] Registered new tserver with Master: 433eed7b49a44911bd5764d0f5a97bcc (127.9.236.2:37111)
I20260929 19:10:16.106423 11888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:52271
I20260929 19:10:16.106695 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 12100
I20260929 19:10:16.106775 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:10:16.106932 10160 external_mini_cluster.cc:1428] Setting key a1e0f62fa44157214a7e90354aebd7a2
I20260929 19:10:16.112602 11938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.106347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:54045 (local address 127.9.236.62:40013)
0929 19:10:16.106390 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:16.106391 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:16.106467 (+ 76us) server_negotiation.cc:407] Connection header received
0929 19:10:16.107083 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:16.107084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:16.107095 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:16.107154 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:16.107523 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.108173 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.108955 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.109214 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.109272 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.110221 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:16.110222 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:16.110222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:16.110229 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:16.111508 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:16.111906 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:16.111907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:16.111907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:16.111950 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:16.112147 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:16.112147 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:16.112148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:16.112272 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:16.112323 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:16.112486 (+ 163us) server_negotiation.cc:299] Negotiation successful
0929 19:10:16.112543 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 19:10:16.112686 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.106257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:40013 (local address 127.9.236.2:54045)
0929 19:10:16.106395 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:16.106407 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:16.106479 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:16.107219 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:16.107221 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:16.107397 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:16.107453 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.107455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.108277 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.108278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.108837 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.108839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.109271 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.109272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.109351 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.109731 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:16.109737 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:16.110118 (+ 381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:16.111591 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:16.111592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:16.111593 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:16.111812 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:16.112015 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:16.112016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:16.112016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:16.112086 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:16.112337 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:16.112338 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:16.112390 (+ 52us) client_negotiation.cc:769] Sending connection context
0929 19:10:16.112467 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 19:10:16.112582 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for kudu/127.9.236.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
I20260929 19:10:16.131949 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:40013
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:34675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:16.218659 12240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:16.218901 12240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:16.218935 12240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:16.220096 12240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:16.220135 12240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:16.220162 12240 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 19:10:16.220177 12240 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 19:10:16.220201 12240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:10:16.221728 12240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:34675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.236.62:40013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:16.221975 12240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:16.222196 12240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:16.227692 12245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:16.227766 12248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:16.227770 12246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:16.228114 12240 server_base.cc:1034] running on GCE node
I20260929 19:10:16.228336 12240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:16.228610 12240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:16.229751 12240 hybrid_clock.cc:648] HybridClock initialized: now 1790709016229724 us; error 43 us; skew 500 ppm
Sep 29 19:10:16 dist-test-slave-wftq krb5kdc[11843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709016, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:16.232383 12240 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:10:16.232911 12240 webserver.cc:460] Webserver started at http://127.9.236.3:45853/ using document root <none> and password file <none>
I20260929 19:10:16.233175 12240 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:16.233229 12240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:16.233376 12240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:16.234549 12240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "77d2dd07d67e4b3496a2860f4c88d944"
format_stamp: "Formatted at 2026-09-29 19:10:16 on dist-test-slave-wftq"
server_key: "6919f5ae74f89a5bbf9072100aa20964"
server_key_iv: "927a9367a61bdbba1e2ba87e64905b14"
server_key_version: "encryptionkey@0"
I20260929 19:10:16.234912 12240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "77d2dd07d67e4b3496a2860f4c88d944"
format_stamp: "Formatted at 2026-09-29 19:10:16 on dist-test-slave-wftq"
server_key: "6919f5ae74f89a5bbf9072100aa20964"
server_key_iv: "927a9367a61bdbba1e2ba87e64905b14"
server_key_version: "encryptionkey@0"
I20260929 19:10:16.236322 12240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:10:16.237178 12255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:16.237392 12240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:16.237481 12240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "77d2dd07d67e4b3496a2860f4c88d944"
format_stamp: "Formatted at 2026-09-29 19:10:16 on dist-test-slave-wftq"
server_key: "6919f5ae74f89a5bbf9072100aa20964"
server_key_iv: "927a9367a61bdbba1e2ba87e64905b14"
server_key_version: "encryptionkey@0"
I20260929 19:10:16.237553 12240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:16.262588 12240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:16.264613 12240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:16.264755 12240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:16.265441 12240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:16.265496 12240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:16.265527 12240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:16.265543 12240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 19:10:16 dist-test-slave-wftq krb5kdc[11843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709016, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:16.273473 12240 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:35771
I20260929 19:10:16.273595 12370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:35771 every 8 connection(s)
I20260929 19:10:16.273939 12240 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:10:16.275411 11938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.265999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:53789 (local address 127.9.236.62:40013)
0929 19:10:16.266063 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:16.266065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:16.266359 (+ 294us) server_negotiation.cc:407] Connection header received
0929 19:10:16.266925 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:16.266926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:16.266937 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:16.266982 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:16.267719 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.268244 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.269350 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.269536 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.269600 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.272299 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:16.272301 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:16.272301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:16.272309 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:16.273874 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:16.274338 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:16.274339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:16.274340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:16.274397 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:16.274751 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:16.274752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:16.274753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:16.274890 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:16.274972 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:16.275271 (+ 299us) server_negotiation.cc:299] Negotiation successful
0929 19:10:16.275341 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260929 19:10:16.275604 12264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.266011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:40013 (local address 127.9.236.3:53789)
0929 19:10:16.266224 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:16.266269 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:16.266830 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:16.267061 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:16.267065 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:16.267415 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:16.267634 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.267638 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.268325 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.268326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.269277 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.269280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.269603 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.269604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.269736 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.270233 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:16.270244 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:16.272209 (+ 1965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:16.273966 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:16.273971 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:16.273974 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:16.274251 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:16.274471 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:16.274473 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:16.274473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:16.274617 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:16.274971 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:16.274975 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:16.275145 (+ 170us) client_negotiation.cc:769] Sending connection context
0929 19:10:16.275237 (+ 92us) client_negotiation.cc:240] Negotiation successful
0929 19:10:16.275402 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":125,"threads_started":1}
I20260929 19:10:16.276114 12371 heartbeater.cc:347] Connected to a master server at 127.9.236.62:40013
I20260929 19:10:16.276225 12371 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:16.276430 12371 heartbeater.cc:511] Master 127.9.236.62:40013 requested a full tablet report, sending...
I20260929 19:10:16.276881 11887 ts_manager.cc:194] Registered new tserver with Master: 77d2dd07d67e4b3496a2860f4c88d944 (127.9.236.3:35771)
I20260929 19:10:16.277437 11887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:53789
I20260929 19:10:16.277535 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 12240
I20260929 19:10:16.277627 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:10:16.277829 10160 external_mini_cluster.cc:1428] Setting key 4333df845ed2b07195ba583a2088234e
I20260929 19:10:16.278662 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:10:16.283874 11938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.277382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:58023 (local address 127.9.236.62:40013)
0929 19:10:16.277428 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:16.277430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:16.277530 (+ 100us) server_negotiation.cc:407] Connection header received
0929 19:10:16.277620 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:16.277621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:16.277631 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:16.277659 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:16.278107 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.278778 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.279825 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.280067 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.280161 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.281227 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:16.281229 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:16.281229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:16.281237 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:16.282718 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:16.283120 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:16.283122 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:16.283122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:16.283181 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:16.283394 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:16.283394 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:16.283395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:16.283497 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:16.283547 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:16.283736 (+ 189us) server_negotiation.cc:299] Negotiation successful
0929 19:10:16.283799 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 19:10:16.283993 12377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.277286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:40013 (local address 127.9.236.3:58023)
0929 19:10:16.277473 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:16.277483 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:16.277554 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:16.277719 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:16.277720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:16.277925 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:16.278001 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.278004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.278945 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.278946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.279740 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.279743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.280173 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.280174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.280262 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.280805 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:16.280809 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:16.281128 (+ 319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:16.282809 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:16.282810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:16.282810 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:16.283040 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:16.283249 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:16.283250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:16.283251 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:16.283320 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:16.283561 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:16.283562 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:16.283626 (+ 64us) client_negotiation.cc:769] Sending connection context
0929 19:10:16.283716 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 19:10:16.283828 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":119,"threads_started":1}
Sep 29 19:10:16 dist-test-slave-wftq krb5kdc[11843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 19:10:16 dist-test-slave-wftq krb5kdc[11843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:16.302107 11938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.295215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34304 (local address 127.9.236.62:40013)
0929 19:10:16.295279 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:16.295281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:16.295289 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:10:16.295314 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:16.295315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:16.295323 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:16.295390 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:16.295919 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.296447 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.297325 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.297518 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.297588 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.299813 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:16.299814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:16.299814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:16.299820 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:16.301092 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:16.301478 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:16.301479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:16.301479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:16.301540 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:16.301698 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:16.301699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:16.301699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:16.301799 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:16.301880 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:16.301964 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 19:10:16.302030 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 19:10:16.303776 11887 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:10:16.305717 11887 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:10:16.313001 12393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.308718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48524 (local address 127.9.236.3:35771)
0929 19:10:16.309041 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:16.309046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:16.309260 (+ 214us) server_negotiation.cc:407] Connection header received
0929 19:10:16.309335 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:16.309338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:16.309373 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:16.309457 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:16.309674 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.310440 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.312085 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.312596 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.312664 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.312725 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:16.312866 (+ 141us) server_negotiation.cc:299] Negotiation successful
0929 19:10:16.312905 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":96,"threads_started":1}
I20260929 19:10:16.312992 12388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.308413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:35771 (local address 127.0.0.1:48524)
0929 19:10:16.309190 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:16.309202 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:16.309267 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:16.309522 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:16.309523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:16.309525 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:16.309584 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.309586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.310523 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.310525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.311973 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.311975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.312677 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.312678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.312760 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.312774 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 19:10:16.312840 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 19:10:16.312900 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"thread_start_us":92,"threads_started":1}
I20260929 19:10:16.313360 12391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.308546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51364 (local address 127.9.236.2:37111)
0929 19:10:16.308751 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:16.308757 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:16.309051 (+ 294us) server_negotiation.cc:407] Connection header received
0929 19:10:16.309263 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:16.309266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:16.309284 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:16.309341 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:16.309605 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.310800 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.312070 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.312801 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.312882 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.312935 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:16.313171 (+ 236us) server_negotiation.cc:299] Negotiation successful
0929 19:10:16.313242 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":94,"threads_started":1}
I20260929 19:10:16.313377 12392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.308701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57370 (local address 127.9.236.1:33007)
0929 19:10:16.308942 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:16.308946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:16.309459 (+ 513us) server_negotiation.cc:407] Connection header received
0929 19:10:16.309506 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:16.309508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:16.309523 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:16.309599 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:16.309805 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.310949 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.312019 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.312800 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.312873 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.312935 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:16.313180 (+ 245us) server_negotiation.cc:299] Negotiation successful
0929 19:10:16.313247 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":121,"threads_started":1}
I20260929 19:10:16.313586 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.308417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:33007 (local address 127.0.0.1:57370)
0929 19:10:16.309368 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:16.309376 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:16.309440 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:16.309661 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:16.309662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:16.309665 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:16.309713 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.309715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.311097 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.311098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.311936 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.311939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.312973 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.312974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.313045 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.313051 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 19:10:16.313456 (+ 405us) client_negotiation.cc:240] Negotiation successful
0929 19:10:16.313515 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":933,"thread_start_us":113,"threads_started":1}
I20260929 19:10:16.313719 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.308411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:37111 (local address 127.0.0.1:51364)
0929 19:10:16.308919 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:16.308964 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:16.309056 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:16.309424 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:16.309427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:16.309433 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:16.309512 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.309515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.310913 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.310914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.312002 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.312004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.312975 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.312976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.313057 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.313061 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:10:16.313614 (+ 553us) client_negotiation.cc:240] Negotiation successful
0929 19:10:16.313662 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"mutex_wait_us":7,"thread_start_us":122,"threads_started":1}
I20260929 19:10:16.314465 12305 tablet_service.cc:1467] Processing CreateTablet for tablet 5acf536ce58f42ae945a86a5178e78f3 (DEFAULT_TABLE table=test-table [id=ddf338d0c23d4560856bf6f1605b03de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:16.314440 12025 tablet_service.cc:1467] Processing CreateTablet for tablet 5acf536ce58f42ae945a86a5178e78f3 (DEFAULT_TABLE table=test-table [id=ddf338d0c23d4560856bf6f1605b03de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:16.314440 12165 tablet_service.cc:1467] Processing CreateTablet for tablet 5acf536ce58f42ae945a86a5178e78f3 (DEFAULT_TABLE table=test-table [id=ddf338d0c23d4560856bf6f1605b03de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:16.314824 12165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5acf536ce58f42ae945a86a5178e78f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:16.314836 12025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5acf536ce58f42ae945a86a5178e78f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:16.314839 12305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5acf536ce58f42ae945a86a5178e78f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:16.317718 12394 tablet_bootstrap.cc:492] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944: Bootstrap starting.
I20260929 19:10:16.318637 12394 tablet_bootstrap.cc:654] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:16.318647 12395 tablet_bootstrap.cc:492] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51: Bootstrap starting.
I20260929 19:10:16.319139 12394 log.cc:824] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:16.319298 12395 tablet_bootstrap.cc:654] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:16.319592 12395 log.cc:824] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:16.319931 12396 tablet_bootstrap.cc:492] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc: Bootstrap starting.
I20260929 19:10:16.320012 12394 tablet_bootstrap.cc:492] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944: No bootstrap required, opened a new log
I20260929 19:10:16.320084 12394 ts_tablet_manager.cc:1397] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:10:16.320314 12395 tablet_bootstrap.cc:492] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51: No bootstrap required, opened a new log
I20260929 19:10:16.320391 12395 ts_tablet_manager.cc:1397] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:10:16.320600 12396 tablet_bootstrap.cc:654] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:16.320928 12396 log.cc:824] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:16.321746 12396 tablet_bootstrap.cc:492] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc: No bootstrap required, opened a new log
I20260929 19:10:16.321815 12396 ts_tablet_manager.cc:1397] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 19:10:16.322225 12394 raft_consensus.cc:348] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } } peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } }
I20260929 19:10:16.322175 12395 raft_consensus.cc:348] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } } peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } }
I20260929 19:10:16.322409 12394 raft_consensus.cc:374] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:16.322422 12395 raft_consensus.cc:374] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:16.322440 12394 raft_consensus.cc:729] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77d2dd07d67e4b3496a2860f4c88d944, State: Initialized, Role: FOLLOWER
I20260929 19:10:16.322465 12395 raft_consensus.cc:729] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6500e0146cf84a229ff00c3d1e1c7f51, State: Initialized, Role: FOLLOWER
I20260929 19:10:16.322572 12394 consensus_queue.cc:260] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } } peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } }
I20260929 19:10:16.322602 12395 consensus_queue.cc:260] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } } peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } }
I20260929 19:10:16.322841 12394 ts_tablet_manager.cc:1428] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:10:16.322887 12395 ts_tablet_manager.cc:1428] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 19:10:16.322985 12371 heartbeater.cc:503] Master 127.9.236.62:40013 was elected leader, sending a full tablet report...
I20260929 19:10:16.323019 12091 heartbeater.cc:503] Master 127.9.236.62:40013 was elected leader, sending a full tablet report...
I20260929 19:10:16.324091 12396 raft_consensus.cc:348] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } } peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } }
I20260929 19:10:16.324244 12396 raft_consensus.cc:374] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:16.324277 12396 raft_consensus.cc:729] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 433eed7b49a44911bd5764d0f5a97bcc, State: Initialized, Role: FOLLOWER
I20260929 19:10:16.324389 12396 consensus_queue.cc:260] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } } peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } }
I20260929 19:10:16.324607 12396 ts_tablet_manager.cc:1428] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 19:10:16.324674 12231 heartbeater.cc:503] Master 127.9.236.62:40013 was elected leader, sending a full tablet report...
W20260929 19:10:16.353061 12232 tablet.cc:2369] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:10:16.429857 12402 raft_consensus.cc:482] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:10:16.429979 12402 raft_consensus.cc:504] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } } peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } }
I20260929 19:10:16.430312 12402 leader_election.cc:288] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77d2dd07d67e4b3496a2860f4c88d944 (127.9.236.3:35771), 6500e0146cf84a229ff00c3d1e1c7f51 (127.9.236.1:33007)
I20260929 19:10:16.433404 12393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.430583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:44233 (local address 127.9.236.3:35771)
0929 19:10:16.430666 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:16.430669 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:16.430678 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:16.430708 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:16.430709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:16.430721 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:16.430782 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:16.431045 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.431577 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.432680 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.433035 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.433095 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.433146 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:16.433286 (+ 140us) server_negotiation.cc:299] Negotiation successful
0929 19:10:16.433326 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 19:10:16.433446 12392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.430627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:42487 (local address 127.9.236.1:33007)
0929 19:10:16.430701 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:16.430704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:16.430819 (+ 115us) server_negotiation.cc:407] Connection header received
0929 19:10:16.430882 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:16.430884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:16.430899 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:16.430969 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:16.431144 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.431645 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.432668 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.433052 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.433153 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.433166 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:16.433330 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 19:10:16.433384 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 19:10:16.433436 12124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.430476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:35771 (local address 127.9.236.2:44233)
0929 19:10:16.430514 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:16.430525 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:16.430569 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:16.430810 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:16.430812 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:16.430815 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:16.430958 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.430961 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.431659 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.431661 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.432571 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.432574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.433138 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.433139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.433212 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.433216 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:10:16.433261 (+ 45us) client_negotiation.cc:240] Negotiation successful
0929 19:10:16.433307 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":69}
I20260929 19:10:16.433746 12403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.430547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:33007 (local address 127.9.236.2:42487)
0929 19:10:16.430741 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:16.430752 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:16.430815 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:16.430979 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:16.430980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:16.430981 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:16.431075 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.431077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.431735 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.431736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.432572 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.432574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.433177 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.433178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.433248 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.433254 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 19:10:16.433315 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 19:10:16.433364 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":1,"thread_start_us":125,"threads_started":1}
I20260929 19:10:16.433876 12322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5acf536ce58f42ae945a86a5178e78f3" candidate_uuid: "433eed7b49a44911bd5764d0f5a97bcc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77d2dd07d67e4b3496a2860f4c88d944" is_pre_election: true
I20260929 19:10:16.434078 12322 raft_consensus.cc:2454] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 433eed7b49a44911bd5764d0f5a97bcc in term 0.
I20260929 19:10:16.434114 12045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5acf536ce58f42ae945a86a5178e78f3" candidate_uuid: "433eed7b49a44911bd5764d0f5a97bcc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" is_pre_election: true
I20260929 19:10:16.434298 12045 raft_consensus.cc:2454] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 433eed7b49a44911bd5764d0f5a97bcc in term 0.
I20260929 19:10:16.434335 12119 leader_election.cc:302] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 433eed7b49a44911bd5764d0f5a97bcc, 77d2dd07d67e4b3496a2860f4c88d944; no voters:
I20260929 19:10:16.434511 12402 raft_consensus.cc:2781] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:10:16.434559 12402 raft_consensus.cc:482] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:10:16.434582 12402 raft_consensus.cc:3037] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:16.435616 12402 raft_consensus.cc:504] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } } peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } }
I20260929 19:10:16.435762 12402 leader_election.cc:288] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc [CANDIDATE]: Term 1 election: Requested vote from peers 77d2dd07d67e4b3496a2860f4c88d944 (127.9.236.3:35771), 6500e0146cf84a229ff00c3d1e1c7f51 (127.9.236.1:33007)
I20260929 19:10:16.435916 12045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5acf536ce58f42ae945a86a5178e78f3" candidate_uuid: "433eed7b49a44911bd5764d0f5a97bcc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6500e0146cf84a229ff00c3d1e1c7f51"
I20260929 19:10:16.435923 12322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5acf536ce58f42ae945a86a5178e78f3" candidate_uuid: "433eed7b49a44911bd5764d0f5a97bcc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77d2dd07d67e4b3496a2860f4c88d944"
I20260929 19:10:16.436008 12045 raft_consensus.cc:3037] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:16.436012 12322 raft_consensus.cc:3037] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:16.436930 12322 raft_consensus.cc:2454] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 433eed7b49a44911bd5764d0f5a97bcc in term 1.
I20260929 19:10:16.436930 12045 raft_consensus.cc:2454] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 433eed7b49a44911bd5764d0f5a97bcc in term 1.
I20260929 19:10:16.437155 12117 leader_election.cc:302] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 433eed7b49a44911bd5764d0f5a97bcc, 6500e0146cf84a229ff00c3d1e1c7f51; no voters:
I20260929 19:10:16.437280 12402 raft_consensus.cc:2781] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:16.437430 12402 raft_consensus.cc:686] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc [term 1 LEADER]: Becoming Leader. State: Replica: 433eed7b49a44911bd5764d0f5a97bcc, State: Running, Role: LEADER
I20260929 19:10:16.437520 12402 consensus_queue.cc:237] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } } peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } }
I20260929 19:10:16.438436 11888 catalog_manager.cc:5495] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc reported cstate change: term changed from 0 to 1, leader changed from <none> to 433eed7b49a44911bd5764d0f5a97bcc (127.9.236.2). New cstate: current_term: 1 leader_uuid: "433eed7b49a44911bd5764d0f5a97bcc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } health_report { overall_health: UNKNOWN } } }
W20260929 19:10:16.441206 12092 tablet.cc:2369] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 29 19:10:16 dist-test-slave-wftq krb5kdc[11843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709015, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:16.481141 12411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.473926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:40013 (local address 127.0.0.1:34306)
0929 19:10:16.474132 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:16.474144 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:16.474229 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:16.474382 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:16.474383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:16.474718 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:16.474770 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.474773 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.475674 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.475675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.476275 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.476277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.476643 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.476644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.476687 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.477263 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:16.477267 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:16.478707 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:16.480191 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:16.480198 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:16.480199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:16.480402 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:16.480576 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:16.480576 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:16.480576 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:16.480639 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:16.480830 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:16.480834 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:16.480875 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 19:10:16.480928 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 19:10:16.481017 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":129,"threads_started":1}
I20260929 19:10:16.481220 11938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.473971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34306 (local address 127.9.236.62:40013)
0929 19:10:16.474050 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:16.474053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:16.474228 (+ 175us) server_negotiation.cc:407] Connection header received
0929 19:10:16.474298 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:16.474299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:16.474313 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:16.474367 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:16.474890 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.475575 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.476368 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.476564 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.476633 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.478790 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:16.478791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:16.478791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:16.478799 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:16.480116 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:16.480480 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:16.480480 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:16.480481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:16.480525 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:16.480687 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:16.480688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:16.480688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:16.480782 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:16.480828 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:16.480947 (+ 119us) server_negotiation.cc:299] Negotiation successful
0929 19:10:16.481062 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":42}
I20260929 19:10:16.483187 11888 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:34306:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 19:10:16.483438 11888 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:10:16.485193 12025 tablet_service.cc:1467] Processing CreateTablet for tablet 7ab8a1460f274a17a6e1a35e20d1b2c0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e1841593345f4b3ba8be1e76234b1acf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 19:10:16.485178 12305 tablet_service.cc:1467] Processing CreateTablet for tablet 7ab8a1460f274a17a6e1a35e20d1b2c0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e1841593345f4b3ba8be1e76234b1acf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 19:10:16.485351 12025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ab8a1460f274a17a6e1a35e20d1b2c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:16.485351 12305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ab8a1460f274a17a6e1a35e20d1b2c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:16.486841 12395 tablet_bootstrap.cc:492] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51: Bootstrap starting.
I20260929 19:10:16.486841 12394 tablet_bootstrap.cc:492] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 77d2dd07d67e4b3496a2860f4c88d944: Bootstrap starting.
I20260929 19:10:16.486943 12165 tablet_service.cc:1467] Processing CreateTablet for tablet 7ab8a1460f274a17a6e1a35e20d1b2c0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e1841593345f4b3ba8be1e76234b1acf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 19:10:16.487085 12165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ab8a1460f274a17a6e1a35e20d1b2c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:16.487399 12395 tablet_bootstrap.cc:654] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:16.487473 12394 tablet_bootstrap.cc:654] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 77d2dd07d67e4b3496a2860f4c88d944: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:16.488168 12395 tablet_bootstrap.cc:492] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51: No bootstrap required, opened a new log
I20260929 19:10:16.488188 12394 tablet_bootstrap.cc:492] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 77d2dd07d67e4b3496a2860f4c88d944: No bootstrap required, opened a new log
I20260929 19:10:16.488221 12395 ts_tablet_manager.cc:1397] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:16.488221 12394 ts_tablet_manager.cc:1397] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 77d2dd07d67e4b3496a2860f4c88d944: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:16.488394 12395 raft_consensus.cc:348] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:16.488394 12394 raft_consensus.cc:348] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:16.488469 12395 raft_consensus.cc:374] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:16.488469 12394 raft_consensus.cc:374] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:16.488489 12394 raft_consensus.cc:729] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77d2dd07d67e4b3496a2860f4c88d944, State: Initialized, Role: FOLLOWER
I20260929 19:10:16.488489 12395 raft_consensus.cc:729] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6500e0146cf84a229ff00c3d1e1c7f51, State: Initialized, Role: FOLLOWER
I20260929 19:10:16.488541 12395 consensus_queue.cc:260] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:16.488540 12394 consensus_queue.cc:260] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 77d2dd07d67e4b3496a2860f4c88d944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:16.488649 12394 ts_tablet_manager.cc:1428] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 77d2dd07d67e4b3496a2860f4c88d944: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:16.488662 12395 ts_tablet_manager.cc:1428] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:16.488754 12396 tablet_bootstrap.cc:492] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 433eed7b49a44911bd5764d0f5a97bcc: Bootstrap starting.
I20260929 19:10:16.488708 12401 tablet_replica.cc:439] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } } }
I20260929 19:10:16.488734 12400 tablet_replica.cc:439] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 77d2dd07d67e4b3496a2860f4c88d944: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } } }
I20260929 19:10:16.488814 12401 tablet_replica.cc:442] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:10:16.488816 12400 tablet_replica.cc:442] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 77d2dd07d67e4b3496a2860f4c88d944: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:10:16.489195 12396 tablet_bootstrap.cc:654] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 433eed7b49a44911bd5764d0f5a97bcc: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:16.489920 12396 tablet_bootstrap.cc:492] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 433eed7b49a44911bd5764d0f5a97bcc: No bootstrap required, opened a new log
I20260929 19:10:16.489966 12396 ts_tablet_manager.cc:1397] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 433eed7b49a44911bd5764d0f5a97bcc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:16.490113 12396 raft_consensus.cc:348] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:16.490193 12396 raft_consensus.cc:374] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:16.490206 12396 raft_consensus.cc:729] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 433eed7b49a44911bd5764d0f5a97bcc, State: Initialized, Role: FOLLOWER
I20260929 19:10:16.490245 12396 consensus_queue.cc:260] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 433eed7b49a44911bd5764d0f5a97bcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:16.490338 12396 ts_tablet_manager.cc:1428] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 433eed7b49a44911bd5764d0f5a97bcc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:16.490404 12402 tablet_replica.cc:439] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 433eed7b49a44911bd5764d0f5a97bcc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } } }
I20260929 19:10:16.490477 12402 tablet_replica.cc:442] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 433eed7b49a44911bd5764d0f5a97bcc: This TxnStatusTablet replica's current role is: FOLLOWER
W20260929 19:10:16.524700 12372 tablet.cc:2369] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:10:16.555020 12401 raft_consensus.cc:482] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:10:16.555150 12401 raft_consensus.cc:504] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:16.555593 12401 leader_election.cc:288] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 433eed7b49a44911bd5764d0f5a97bcc (127.9.236.2:37111), 77d2dd07d67e4b3496a2860f4c88d944 (127.9.236.3:35771)
I20260929 19:10:16.558866 12416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.555771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:37111 (local address 127.9.236.1:38863)
0929 19:10:16.555991 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:16.556003 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:16.556090 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:16.556221 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:16.556222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:16.556225 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:16.556316 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.556318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.557066 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.557067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.557988 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.557990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.558538 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.558540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.558603 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.558607 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:10:16.558712 (+ 105us) client_negotiation.cc:240] Negotiation successful
0929 19:10:16.558762 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":22,"thread_start_us":68,"threads_started":1}
I20260929 19:10:16.558840 12391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.555902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:38863 (local address 127.9.236.2:37111)
0929 19:10:16.555953 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:16.555955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:16.556079 (+ 124us) server_negotiation.cc:407] Connection header received
0929 19:10:16.556150 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:16.556151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:16.556164 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:16.556214 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:16.556370 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.556986 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.558074 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.558449 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.558508 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.558520 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:16.558723 (+ 203us) server_negotiation.cc:299] Negotiation successful
0929 19:10:16.558766 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:10:16.558957 12393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.555873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:47269 (local address 127.9.236.3:35771)
0929 19:10:16.555968 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:16.555971 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:16.555985 (+ 14us) server_negotiation.cc:407] Connection header received
0929 19:10:16.556066 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:16.556067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:16.556077 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:16.556125 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:16.556263 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.556796 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.557978 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.558329 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.558388 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.558398 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:16.558839 (+ 441us) server_negotiation.cc:299] Negotiation successful
0929 19:10:16.558875 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 19:10:16.558974 12415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.555771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:35771 (local address 127.9.236.1:47269)
0929 19:10:16.555935 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:16.555947 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:16.556006 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:16.556135 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:16.556136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:16.556139 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:16.556207 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.556209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.556865 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.556866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.557904 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.557907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.558710 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.558712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.558768 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.558773 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:10:16.558824 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 19:10:16.558879 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":22,"thread_start_us":83,"threads_started":1}
I20260929 19:10:16.559464 12322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab8a1460f274a17a6e1a35e20d1b2c0" candidate_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77d2dd07d67e4b3496a2860f4c88d944" is_pre_election: true
I20260929 19:10:16.559588 12322 raft_consensus.cc:2454] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6500e0146cf84a229ff00c3d1e1c7f51 in term 0.
I20260929 19:10:16.559559 12185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab8a1460f274a17a6e1a35e20d1b2c0" candidate_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "433eed7b49a44911bd5764d0f5a97bcc" is_pre_election: true
I20260929 19:10:16.559682 12185 raft_consensus.cc:2454] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6500e0146cf84a229ff00c3d1e1c7f51 in term 0.
I20260929 19:10:16.559777 11979 leader_election.cc:302] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6500e0146cf84a229ff00c3d1e1c7f51, 77d2dd07d67e4b3496a2860f4c88d944; no voters:
I20260929 19:10:16.559937 12401 raft_consensus.cc:2781] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:10:16.559984 12401 raft_consensus.cc:482] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:10:16.560005 12401 raft_consensus.cc:3037] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:16.560809 12401 raft_consensus.cc:504] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:16.560956 12401 leader_election.cc:288] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51 [CANDIDATE]: Term 1 election: Requested vote from peers 433eed7b49a44911bd5764d0f5a97bcc (127.9.236.2:37111), 77d2dd07d67e4b3496a2860f4c88d944 (127.9.236.3:35771)
I20260929 19:10:16.561211 12322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab8a1460f274a17a6e1a35e20d1b2c0" candidate_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77d2dd07d67e4b3496a2860f4c88d944"
I20260929 19:10:16.561177 12185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab8a1460f274a17a6e1a35e20d1b2c0" candidate_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "433eed7b49a44911bd5764d0f5a97bcc"
I20260929 19:10:16.561287 12322 raft_consensus.cc:3037] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:16.561290 12185 raft_consensus.cc:3037] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:16.561945 12322 raft_consensus.cc:2454] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 77d2dd07d67e4b3496a2860f4c88d944 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6500e0146cf84a229ff00c3d1e1c7f51 in term 1.
I20260929 19:10:16.562033 12185 raft_consensus.cc:2454] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 433eed7b49a44911bd5764d0f5a97bcc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6500e0146cf84a229ff00c3d1e1c7f51 in term 1.
I20260929 19:10:16.562093 11979 leader_election.cc:302] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6500e0146cf84a229ff00c3d1e1c7f51, 77d2dd07d67e4b3496a2860f4c88d944; no voters:
I20260929 19:10:16.562201 12401 raft_consensus.cc:2781] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:16.562333 12401 raft_consensus.cc:686] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 1 LEADER]: Becoming Leader. State: Replica: 6500e0146cf84a229ff00c3d1e1c7f51, State: Running, Role: LEADER
I20260929 19:10:16.562425 12401 consensus_queue.cc:237] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:16.562872 12417 tablet_replica.cc:439] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51: TxnStatusTablet state changed. Reason: New leader 6500e0146cf84a229ff00c3d1e1c7f51. Latest consensus state: current_term: 1 leader_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } } }
I20260929 19:10:16.562981 12417 tablet_replica.cc:442] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51: This TxnStatusTablet replica's current role is: LEADER
I20260929 19:10:16.563220 12419 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 19:10:16.563247 11888 catalog_manager.cc:5495] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6500e0146cf84a229ff00c3d1e1c7f51 (127.9.236.1). New cstate: current_term: 1 leader_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } health_report { overall_health: UNKNOWN } } }
I20260929 19:10:16.563347 12419 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 19:10:16.583389 12392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.580514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57384 (local address 127.9.236.1:33007)
0929 19:10:16.580556 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:16.580558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:16.580596 (+ 38us) server_negotiation.cc:407] Connection header received
0929 19:10:16.580700 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:16.580701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:16.580718 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:16.580785 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:16.580939 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.581478 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.582656 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.582837 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.582892 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.583090 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:16.583195 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:16.583253 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:16.583308 (+ 55us) server_negotiation.cc:299] Negotiation successful
0929 19:10:16.583327 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:10:16.583487 12411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.580462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:33007 (local address 127.0.0.1:57384)
0929 19:10:16.580541 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:16.580554 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:16.580613 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:16.580792 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:16.580793 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:16.580794 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:10:16.580866 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.580870 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.581550 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.581551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.582571 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.582574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.582910 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.582911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.582983 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.583001 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:10:16.583252 (+ 251us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:10:16.583258 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 19:10:16.583307 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 19:10:16.583365 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260929 19:10:16.585211 12185 raft_consensus.cc:1261] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 433eed7b49a44911bd5764d0f5a97bcc [term 1 FOLLOWER]: Refusing update from remote peer 6500e0146cf84a229ff00c3d1e1c7f51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:16.585280 12322 raft_consensus.cc:1261] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 77d2dd07d67e4b3496a2860f4c88d944 [term 1 FOLLOWER]: Refusing update from remote peer 6500e0146cf84a229ff00c3d1e1c7f51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:16.585553 12417 consensus_queue.cc:1035] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:16.585666 12401 consensus_queue.cc:1035] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:16.586825 12400 tablet_replica.cc:439] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 77d2dd07d67e4b3496a2860f4c88d944: TxnStatusTablet state changed. Reason: New leader 6500e0146cf84a229ff00c3d1e1c7f51. Latest consensus state: current_term: 1 leader_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } } }
I20260929 19:10:16.586912 12400 tablet_replica.cc:442] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 77d2dd07d67e4b3496a2860f4c88d944: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:10:16.587172 12402 tablet_replica.cc:439] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 433eed7b49a44911bd5764d0f5a97bcc: TxnStatusTablet state changed. Reason: New leader 6500e0146cf84a229ff00c3d1e1c7f51. Latest consensus state: current_term: 1 leader_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } } }
I20260929 19:10:16.587275 12402 tablet_replica.cc:442] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 433eed7b49a44911bd5764d0f5a97bcc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:10:16.587795 12401 tablet_replica.cc:439] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } } }
I20260929 19:10:16.587886 12401 tablet_replica.cc:442] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51: This TxnStatusTablet replica's current role is: LEADER
I20260929 19:10:16.587956 12422 mvcc.cc:205] Tried to move back new op lower bound from 7334744131930521600 to 7334744131840385024. Current Snapshot: MvccSnapshot[applied={T|T < 7334744131930521600}]
I20260929 19:10:16.588521 12423 mvcc.cc:205] Tried to move back new op lower bound from 7334744131930521600 to 7334744131840385024. Current Snapshot: MvccSnapshot[applied={T|T < 7334744131930521600}]
I20260929 19:10:16.588603 12400 tablet_replica.cc:439] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 77d2dd07d67e4b3496a2860f4c88d944: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } } }
I20260929 19:10:16.588673 12400 tablet_replica.cc:442] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 77d2dd07d67e4b3496a2860f4c88d944: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:10:16.589041 12417 tablet_replica.cc:439] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } } }
I20260929 19:10:16.589139 12417 tablet_replica.cc:442] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 6500e0146cf84a229ff00c3d1e1c7f51: This TxnStatusTablet replica's current role is: LEADER
I20260929 19:10:16.589504 12402 tablet_replica.cc:439] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 433eed7b49a44911bd5764d0f5a97bcc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } } }
I20260929 19:10:16.589591 12402 tablet_replica.cc:442] T 7ab8a1460f274a17a6e1a35e20d1b2c0 P 433eed7b49a44911bd5764d0f5a97bcc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:10:16.590214 12425 mvcc.cc:205] Tried to move back new op lower bound from 7334744131930521600 to 7334744131840385024. Current Snapshot: MvccSnapshot[applied={T|T < 7334744131930521600}]
I20260929 19:10:16.593165 12391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.590936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51376 (local address 127.9.236.2:37111)
0929 19:10:16.590985 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:16.590987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:16.590995 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:10:16.591027 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:16.591027 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:16.591046 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:16.591133 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:16.591244 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.591716 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.592465 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.592665 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.592717 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.592853 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:16.592949 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:16.593011 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:16.593047 (+ 36us) server_negotiation.cc:299] Negotiation successful
0929 19:10:16.593068 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260929 19:10:16.598320 12392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.595805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:39841 (local address 127.9.236.1:33007)
0929 19:10:16.595844 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:16.595846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:16.595869 (+ 23us) server_negotiation.cc:407] Connection header received
0929 19:10:16.595937 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:16.595937 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:16.595945 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:16.595990 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:16.596129 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.596549 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.597702 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.597875 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.597938 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.598135 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:16.598160 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:16.598197 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:16.598262 (+ 65us) server_negotiation.cc:299] Negotiation successful
0929 19:10:16.598274 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 19:10:16.598390 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.595733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:33007 (local address 127.9.236.2:39841)
0929 19:10:16.595816 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:16.595829 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:16.595880 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:16.596006 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:16.596007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:16.596008 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:10:16.596077 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.596079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.596612 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.596612 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.597604 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.597606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.597952 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.597953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.598022 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.598042 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:10:16.598215 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:10:16.598217 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:10:16.598263 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 19:10:16.598309 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260929 19:10:16.600857 12322 raft_consensus.cc:1261] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944 [term 1 FOLLOWER]: Refusing update from remote peer 433eed7b49a44911bd5764d0f5a97bcc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:16.601004 12045 raft_consensus.cc:1261] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 1 FOLLOWER]: Refusing update from remote peer 433eed7b49a44911bd5764d0f5a97bcc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:16.601094 12402 consensus_queue.cc:1035] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc [LEADER]: Connected to new peer: Peer: permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:16.601291 12402 consensus_queue.cc:1035] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:16.611559 12391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.608542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:37353 (local address 127.9.236.2:37111)
0929 19:10:16.608586 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:16.608588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:16.608675 (+ 87us) server_negotiation.cc:407] Connection header received
0929 19:10:16.608737 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:16.608738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:16.608747 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:16.608794 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:16.608931 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.609428 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.610951 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.611125 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.611185 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.611365 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:16.611393 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:16.611426 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:16.611494 (+ 68us) server_negotiation.cc:299] Negotiation successful
0929 19:10:16.611509 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 19:10:16.611647 12439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:16.608450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:37111 (local address 127.9.236.1:37353)
0929 19:10:16.608602 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:16.608614 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:16.608687 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:16.608801 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:16.608802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:16.608814 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:10:16.608873 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.608875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.609502 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.609503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.610871 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:16.610874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:16.611198 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:16.611199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:16.611272 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:16.611290 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:10:16.611449 (+ 159us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:10:16.611450 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 19:10:16.611494 (+ 44us) client_negotiation.cc:240] Negotiation successful
0929 19:10:16.611541 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
I20260929 19:10:16.620621 11888 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:16.620726 11888 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:16.621405 11888 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417: Sending DeleteTablet for 3 replicas of tablet 5acf536ce58f42ae945a86a5178e78f3
I20260929 19:10:16.621719 12165 tablet_service.cc:1514] Processing DeleteTablet for tablet 5acf536ce58f42ae945a86a5178e78f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:16 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:51364
I20260929 19:10:16.621978 12305 tablet_service.cc:1514] Processing DeleteTablet for tablet 5acf536ce58f42ae945a86a5178e78f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:16 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:48524
I20260929 19:10:16.622212 11888 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:10:16.622337 11888 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:10:16.622646 12025 tablet_service.cc:1514] Processing DeleteTablet for tablet 5acf536ce58f42ae945a86a5178e78f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:16 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:57370
I20260929 19:10:16.623566 12305 tablet_service.cc:1467] Processing CreateTablet for tablet e18db6e1ca5f45f2ac75d7e3c820598e (DEFAULT_TABLE table=test-table [id=dadb2c2b0a684885bf9e9bd0add354c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:16.623696 12305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e18db6e1ca5f45f2ac75d7e3c820598e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:16.624881 12025 tablet_service.cc:1467] Processing CreateTablet for tablet e18db6e1ca5f45f2ac75d7e3c820598e (DEFAULT_TABLE table=test-table [id=dadb2c2b0a684885bf9e9bd0add354c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:16.624989 12025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e18db6e1ca5f45f2ac75d7e3c820598e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:16.625958 12395 tablet_bootstrap.cc:492] T e18db6e1ca5f45f2ac75d7e3c820598e P 6500e0146cf84a229ff00c3d1e1c7f51: Bootstrap starting.
I20260929 19:10:16.625998 12165 tablet_service.cc:1467] Processing CreateTablet for tablet e18db6e1ca5f45f2ac75d7e3c820598e (DEFAULT_TABLE table=test-table [id=dadb2c2b0a684885bf9e9bd0add354c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:16.626091 12165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e18db6e1ca5f45f2ac75d7e3c820598e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:16.626344 12395 tablet_bootstrap.cc:654] T e18db6e1ca5f45f2ac75d7e3c820598e P 6500e0146cf84a229ff00c3d1e1c7f51: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:16.627281 12396 tablet_bootstrap.cc:492] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc: Bootstrap starting.
I20260929 19:10:16.627336 12394 tablet_bootstrap.cc:492] T e18db6e1ca5f45f2ac75d7e3c820598e P 77d2dd07d67e4b3496a2860f4c88d944: Bootstrap starting.
I20260929 19:10:16.627722 12394 tablet_bootstrap.cc:654] T e18db6e1ca5f45f2ac75d7e3c820598e P 77d2dd07d67e4b3496a2860f4c88d944: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:16.627875 12396 tablet_bootstrap.cc:654] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:16.628334 12444 tablet_replica.cc:330] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51: stopping tablet replica
I20260929 19:10:16.628449 12444 raft_consensus.cc:2229] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:16.628509 12444 raft_consensus.cc:2258] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:16.628635 12443 tablet_replica.cc:330] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944: stopping tablet replica
I20260929 19:10:16.628733 12443 raft_consensus.cc:2229] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:16.628789 12443 raft_consensus.cc:2258] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:16.629022 12442 tablet_replica.cc:330] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc: stopping tablet replica
I20260929 19:10:16.629164 12442 raft_consensus.cc:2229] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:10:16.629396 12443 ts_tablet_manager.cc:1905] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:16.629460 12444 ts_tablet_manager.cc:1905] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:16.630767 12442 raft_consensus.cc:2258] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:16.631021 12444 ts_tablet_manager.cc:1918] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:10:16.631088 12444 log.cc:1196] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-0/wal/wals/5acf536ce58f42ae945a86a5178e78f3
I20260929 19:10:16.631304 12443 ts_tablet_manager.cc:1918] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:10:16.631362 12444 ts_tablet_manager.cc:1939] T 5acf536ce58f42ae945a86a5178e78f3 P 6500e0146cf84a229ff00c3d1e1c7f51: Deleting consensus metadata
I20260929 19:10:16.631381 12443 log.cc:1196] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/5acf536ce58f42ae945a86a5178e78f3
I20260929 19:10:16.631644 12443 ts_tablet_manager.cc:1939] T 5acf536ce58f42ae945a86a5178e78f3 P 77d2dd07d67e4b3496a2860f4c88d944: Deleting consensus metadata
I20260929 19:10:16.631754 11873 catalog_manager.cc:4841] TS 6500e0146cf84a229ff00c3d1e1c7f51 (127.9.236.1:33007): tablet 5acf536ce58f42ae945a86a5178e78f3 (table test-table [id=ddf338d0c23d4560856bf6f1605b03de]) successfully deleted
I20260929 19:10:16.631991 12442 ts_tablet_manager.cc:1905] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:16.633280 12442 ts_tablet_manager.cc:1918] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:10:16.633389 12442 log.cc:1196] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/5acf536ce58f42ae945a86a5178e78f3
I20260929 19:10:16.633528 12395 tablet_bootstrap.cc:492] T e18db6e1ca5f45f2ac75d7e3c820598e P 6500e0146cf84a229ff00c3d1e1c7f51: No bootstrap required, opened a new log
I20260929 19:10:16.633582 12395 ts_tablet_manager.cc:1397] T e18db6e1ca5f45f2ac75d7e3c820598e P 6500e0146cf84a229ff00c3d1e1c7f51: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260929 19:10:16.633620 12442 ts_tablet_manager.cc:1939] T 5acf536ce58f42ae945a86a5178e78f3 P 433eed7b49a44911bd5764d0f5a97bcc: Deleting consensus metadata
I20260929 19:10:16.633746 12395 raft_consensus.cc:348] T e18db6e1ca5f45f2ac75d7e3c820598e P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:16.633812 12395 raft_consensus.cc:374] T e18db6e1ca5f45f2ac75d7e3c820598e P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:16.633829 12395 raft_consensus.cc:729] T e18db6e1ca5f45f2ac75d7e3c820598e P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6500e0146cf84a229ff00c3d1e1c7f51, State: Initialized, Role: FOLLOWER
I20260929 19:10:16.633869 12395 consensus_queue.cc:260] T e18db6e1ca5f45f2ac75d7e3c820598e P 6500e0146cf84a229ff00c3d1e1c7f51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:16.633950 12395 ts_tablet_manager.cc:1428] T e18db6e1ca5f45f2ac75d7e3c820598e P 6500e0146cf84a229ff00c3d1e1c7f51: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:16.633971 11874 catalog_manager.cc:4841] TS 433eed7b49a44911bd5764d0f5a97bcc (127.9.236.2:37111): tablet 5acf536ce58f42ae945a86a5178e78f3 (table test-table [id=ddf338d0c23d4560856bf6f1605b03de]) successfully deleted
I20260929 19:10:16.634459 12396 tablet_bootstrap.cc:492] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc: No bootstrap required, opened a new log
I20260929 19:10:16.634510 12396 ts_tablet_manager.cc:1397] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260929 19:10:16.634660 12396 raft_consensus.cc:348] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:16.634716 12396 raft_consensus.cc:374] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:16.634733 12396 raft_consensus.cc:729] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 433eed7b49a44911bd5764d0f5a97bcc, State: Initialized, Role: FOLLOWER
I20260929 19:10:16.634768 12396 consensus_queue.cc:260] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:16.634846 12396 ts_tablet_manager.cc:1428] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:16.637478 11875 catalog_manager.cc:4841] TS 77d2dd07d67e4b3496a2860f4c88d944 (127.9.236.3:35771): tablet 5acf536ce58f42ae945a86a5178e78f3 (table test-table [id=ddf338d0c23d4560856bf6f1605b03de]) successfully deleted
I20260929 19:10:16.637748 12394 tablet_bootstrap.cc:492] T e18db6e1ca5f45f2ac75d7e3c820598e P 77d2dd07d67e4b3496a2860f4c88d944: No bootstrap required, opened a new log
I20260929 19:10:16.637791 12394 ts_tablet_manager.cc:1397] T e18db6e1ca5f45f2ac75d7e3c820598e P 77d2dd07d67e4b3496a2860f4c88d944: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260929 19:10:16.637925 12394 raft_consensus.cc:348] T e18db6e1ca5f45f2ac75d7e3c820598e P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:16.637981 12394 raft_consensus.cc:374] T e18db6e1ca5f45f2ac75d7e3c820598e P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:16.637997 12394 raft_consensus.cc:729] T e18db6e1ca5f45f2ac75d7e3c820598e P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77d2dd07d67e4b3496a2860f4c88d944, State: Initialized, Role: FOLLOWER
I20260929 19:10:16.638034 12394 consensus_queue.cc:260] T e18db6e1ca5f45f2ac75d7e3c820598e P 77d2dd07d67e4b3496a2860f4c88d944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:16.638118 12394 ts_tablet_manager.cc:1428] T e18db6e1ca5f45f2ac75d7e3c820598e P 77d2dd07d67e4b3496a2860f4c88d944: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:16.757012 12436 raft_consensus.cc:482] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:10:16.757144 12436 raft_consensus.cc:504] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:16.757320 12436 leader_election.cc:288] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6500e0146cf84a229ff00c3d1e1c7f51 (127.9.236.1:33007), 77d2dd07d67e4b3496a2860f4c88d944 (127.9.236.3:35771)
I20260929 19:10:16.757508 12045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18db6e1ca5f45f2ac75d7e3c820598e" candidate_uuid: "433eed7b49a44911bd5764d0f5a97bcc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" is_pre_election: true
I20260929 19:10:16.757601 12322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18db6e1ca5f45f2ac75d7e3c820598e" candidate_uuid: "433eed7b49a44911bd5764d0f5a97bcc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77d2dd07d67e4b3496a2860f4c88d944" is_pre_election: true
I20260929 19:10:16.757606 12045 raft_consensus.cc:2454] T e18db6e1ca5f45f2ac75d7e3c820598e P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 433eed7b49a44911bd5764d0f5a97bcc in term 0.
I20260929 19:10:16.757679 12322 raft_consensus.cc:2454] T e18db6e1ca5f45f2ac75d7e3c820598e P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 433eed7b49a44911bd5764d0f5a97bcc in term 0.
I20260929 19:10:16.757844 12119 leader_election.cc:302] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 433eed7b49a44911bd5764d0f5a97bcc, 77d2dd07d67e4b3496a2860f4c88d944; no voters:
I20260929 19:10:16.757977 12436 raft_consensus.cc:2781] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:10:16.758100 12436 raft_consensus.cc:482] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:10:16.758149 12436 raft_consensus.cc:3037] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:16.759030 12436 raft_consensus.cc:504] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:16.759227 12436 leader_election.cc:288] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc [CANDIDATE]: Term 1 election: Requested vote from peers 6500e0146cf84a229ff00c3d1e1c7f51 (127.9.236.1:33007), 77d2dd07d67e4b3496a2860f4c88d944 (127.9.236.3:35771)
I20260929 19:10:16.759366 12045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18db6e1ca5f45f2ac75d7e3c820598e" candidate_uuid: "433eed7b49a44911bd5764d0f5a97bcc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6500e0146cf84a229ff00c3d1e1c7f51"
I20260929 19:10:16.759415 12322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18db6e1ca5f45f2ac75d7e3c820598e" candidate_uuid: "433eed7b49a44911bd5764d0f5a97bcc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77d2dd07d67e4b3496a2860f4c88d944"
I20260929 19:10:16.759469 12322 raft_consensus.cc:3037] T e18db6e1ca5f45f2ac75d7e3c820598e P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:16.759469 12045 raft_consensus.cc:3037] T e18db6e1ca5f45f2ac75d7e3c820598e P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:16.760308 12045 raft_consensus.cc:2454] T e18db6e1ca5f45f2ac75d7e3c820598e P 6500e0146cf84a229ff00c3d1e1c7f51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 433eed7b49a44911bd5764d0f5a97bcc in term 1.
I20260929 19:10:16.760308 12322 raft_consensus.cc:2454] T e18db6e1ca5f45f2ac75d7e3c820598e P 77d2dd07d67e4b3496a2860f4c88d944 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 433eed7b49a44911bd5764d0f5a97bcc in term 1.
I20260929 19:10:16.760510 12117 leader_election.cc:302] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 433eed7b49a44911bd5764d0f5a97bcc, 6500e0146cf84a229ff00c3d1e1c7f51; no voters:
I20260929 19:10:16.760648 12436 raft_consensus.cc:2781] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:16.760694 12436 raft_consensus.cc:686] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc [term 1 LEADER]: Becoming Leader. State: Replica: 433eed7b49a44911bd5764d0f5a97bcc, State: Running, Role: LEADER
I20260929 19:10:16.760744 12436 consensus_queue.cc:237] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:16.761405 11887 catalog_manager.cc:5495] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc reported cstate change: term changed from 0 to 1, leader changed from <none> to 433eed7b49a44911bd5764d0f5a97bcc (127.9.236.2). New cstate: current_term: 1 leader_uuid: "433eed7b49a44911bd5764d0f5a97bcc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } health_report { overall_health: UNKNOWN } } }
I20260929 19:10:16.777565 12322 raft_consensus.cc:1261] T e18db6e1ca5f45f2ac75d7e3c820598e P 77d2dd07d67e4b3496a2860f4c88d944 [term 1 FOLLOWER]: Refusing update from remote peer 433eed7b49a44911bd5764d0f5a97bcc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:16.777617 12045 raft_consensus.cc:1261] T e18db6e1ca5f45f2ac75d7e3c820598e P 6500e0146cf84a229ff00c3d1e1c7f51 [term 1 FOLLOWER]: Refusing update from remote peer 433eed7b49a44911bd5764d0f5a97bcc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:16.777853 12404 consensus_queue.cc:1035] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc [LEADER]: Connected to new peer: Peer: permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:16.778002 12436 consensus_queue.cc:1035] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 19:10:16.782943 12425 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 19:10:18.302436 11887 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:18.302572 11887 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:18.303544 11887 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417: Sending DeleteTablet for 3 replicas of tablet e18db6e1ca5f45f2ac75d7e3c820598e
I20260929 19:10:18.303838 12165 tablet_service.cc:1514] Processing DeleteTablet for tablet e18db6e1ca5f45f2ac75d7e3c820598e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:18 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:51364
I20260929 19:10:18.303902 12305 tablet_service.cc:1514] Processing DeleteTablet for tablet e18db6e1ca5f45f2ac75d7e3c820598e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:18 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:48524
I20260929 19:10:18.303917 12024 tablet_service.cc:1514] Processing DeleteTablet for tablet e18db6e1ca5f45f2ac75d7e3c820598e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:18 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:57370
I20260929 19:10:18.304177 12473 tablet_replica.cc:330] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc: stopping tablet replica
I20260929 19:10:18.304203 12474 tablet_replica.cc:330] T e18db6e1ca5f45f2ac75d7e3c820598e P 77d2dd07d67e4b3496a2860f4c88d944: stopping tablet replica
I20260929 19:10:18.304239 12473 raft_consensus.cc:2229] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:10:18.304265 12474 raft_consensus.cc:2229] T e18db6e1ca5f45f2ac75d7e3c820598e P 77d2dd07d67e4b3496a2860f4c88d944 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:18.304311 12473 raft_consensus.cc:2258] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:18.304318 12474 raft_consensus.cc:2258] T e18db6e1ca5f45f2ac75d7e3c820598e P 77d2dd07d67e4b3496a2860f4c88d944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:18.304441 12475 tablet_replica.cc:330] T e18db6e1ca5f45f2ac75d7e3c820598e P 6500e0146cf84a229ff00c3d1e1c7f51: stopping tablet replica
I20260929 19:10:18.304493 12475 raft_consensus.cc:2229] T e18db6e1ca5f45f2ac75d7e3c820598e P 6500e0146cf84a229ff00c3d1e1c7f51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:18.304522 12475 raft_consensus.cc:2258] T e18db6e1ca5f45f2ac75d7e3c820598e P 6500e0146cf84a229ff00c3d1e1c7f51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:18.304677 12474 ts_tablet_manager.cc:1905] T e18db6e1ca5f45f2ac75d7e3c820598e P 77d2dd07d67e4b3496a2860f4c88d944: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:18.306363 12475 ts_tablet_manager.cc:1905] T e18db6e1ca5f45f2ac75d7e3c820598e P 6500e0146cf84a229ff00c3d1e1c7f51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:18.306908 12473 ts_tablet_manager.cc:1905] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:18.307816 12475 ts_tablet_manager.cc:1918] T e18db6e1ca5f45f2ac75d7e3c820598e P 6500e0146cf84a229ff00c3d1e1c7f51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 19:10:18.307869 12475 log.cc:1196] T e18db6e1ca5f45f2ac75d7e3c820598e P 6500e0146cf84a229ff00c3d1e1c7f51: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-0/wal/wals/e18db6e1ca5f45f2ac75d7e3c820598e
I20260929 19:10:18.308059 12475 ts_tablet_manager.cc:1939] T e18db6e1ca5f45f2ac75d7e3c820598e P 6500e0146cf84a229ff00c3d1e1c7f51: Deleting consensus metadata
I20260929 19:10:18.308192 12473 ts_tablet_manager.cc:1918] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 19:10:18.308241 12473 log.cc:1196] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/e18db6e1ca5f45f2ac75d7e3c820598e
I20260929 19:10:18.308472 12473 ts_tablet_manager.cc:1939] T e18db6e1ca5f45f2ac75d7e3c820598e P 433eed7b49a44911bd5764d0f5a97bcc: Deleting consensus metadata
I20260929 19:10:18.308665 12474 ts_tablet_manager.cc:1918] T e18db6e1ca5f45f2ac75d7e3c820598e P 77d2dd07d67e4b3496a2860f4c88d944: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 19:10:18.308681 11873 catalog_manager.cc:4841] TS 6500e0146cf84a229ff00c3d1e1c7f51 (127.9.236.1:33007): tablet e18db6e1ca5f45f2ac75d7e3c820598e (table test-table [id=dadb2c2b0a684885bf9e9bd0add354c6]) successfully deleted
I20260929 19:10:18.308729 12474 log.cc:1196] T e18db6e1ca5f45f2ac75d7e3c820598e P 77d2dd07d67e4b3496a2860f4c88d944: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/e18db6e1ca5f45f2ac75d7e3c820598e
I20260929 19:10:18.308949 12474 ts_tablet_manager.cc:1939] T e18db6e1ca5f45f2ac75d7e3c820598e P 77d2dd07d67e4b3496a2860f4c88d944: Deleting consensus metadata
I20260929 19:10:18.309131 11874 catalog_manager.cc:4841] TS 433eed7b49a44911bd5764d0f5a97bcc (127.9.236.2:37111): tablet e18db6e1ca5f45f2ac75d7e3c820598e (table test-table [id=dadb2c2b0a684885bf9e9bd0add354c6]) successfully deleted
I20260929 19:10:18.309266 11875 catalog_manager.cc:4841] TS 77d2dd07d67e4b3496a2860f4c88d944 (127.9.236.3:35771): tablet e18db6e1ca5f45f2ac75d7e3c820598e (table test-table [id=dadb2c2b0a684885bf9e9bd0add354c6]) successfully deleted
Sep 29 19:10:18 dist-test-slave-wftq krb5kdc[11843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709018, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 19:10:18 dist-test-slave-wftq krb5kdc[11843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709018, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:18.331465 12483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:18.324244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34320 (local address 127.9.236.62:40013)
0929 19:10:18.324397 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:18.324399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:18.324417 (+ 18us) server_negotiation.cc:407] Connection header received
0929 19:10:18.324521 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:18.324522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:18.324535 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:18.324606 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:18.325214 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:18.325766 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:18.326489 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:18.326696 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:18.326749 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:18.329125 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:18.329127 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:18.329127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:18.329135 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:18.330509 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:18.330832 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:18.330832 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:18.330833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:18.330880 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:18.331035 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:18.331036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:18.331036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:18.331142 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:18.331222 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:18.331295 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 19:10:18.331360 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260929 19:10:18.333078 11887 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:10:18.333271 11887 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:10:18.334695 12024 tablet_service.cc:1467] Processing CreateTablet for tablet b4113be2795944f59de0846cdec3002b (DEFAULT_TABLE table=test-table [id=8cce6c2a2f534a74b06c8c4847cd4b48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:18.334756 12165 tablet_service.cc:1467] Processing CreateTablet for tablet b4113be2795944f59de0846cdec3002b (DEFAULT_TABLE table=test-table [id=8cce6c2a2f534a74b06c8c4847cd4b48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:18.334772 12305 tablet_service.cc:1467] Processing CreateTablet for tablet b4113be2795944f59de0846cdec3002b (DEFAULT_TABLE table=test-table [id=8cce6c2a2f534a74b06c8c4847cd4b48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:18.334863 12165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4113be2795944f59de0846cdec3002b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:18.334862 12024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4113be2795944f59de0846cdec3002b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:18.334930 12305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4113be2795944f59de0846cdec3002b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:18.336189 12484 tablet_bootstrap.cc:492] T b4113be2795944f59de0846cdec3002b P 77d2dd07d67e4b3496a2860f4c88d944: Bootstrap starting.
I20260929 19:10:18.336498 12485 tablet_bootstrap.cc:492] T b4113be2795944f59de0846cdec3002b P 433eed7b49a44911bd5764d0f5a97bcc: Bootstrap starting.
I20260929 19:10:18.336715 12484 tablet_bootstrap.cc:654] T b4113be2795944f59de0846cdec3002b P 77d2dd07d67e4b3496a2860f4c88d944: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:18.336864 12485 tablet_bootstrap.cc:654] T b4113be2795944f59de0846cdec3002b P 433eed7b49a44911bd5764d0f5a97bcc: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:18.337196 12486 tablet_bootstrap.cc:492] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51: Bootstrap starting.
I20260929 19:10:18.337518 12484 tablet_bootstrap.cc:492] T b4113be2795944f59de0846cdec3002b P 77d2dd07d67e4b3496a2860f4c88d944: No bootstrap required, opened a new log
I20260929 19:10:18.337513 12485 tablet_bootstrap.cc:492] T b4113be2795944f59de0846cdec3002b P 433eed7b49a44911bd5764d0f5a97bcc: No bootstrap required, opened a new log
I20260929 19:10:18.337558 12486 tablet_bootstrap.cc:654] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:18.337571 12485 ts_tablet_manager.cc:1397] T b4113be2795944f59de0846cdec3002b P 433eed7b49a44911bd5764d0f5a97bcc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:18.337571 12484 ts_tablet_manager.cc:1397] T b4113be2795944f59de0846cdec3002b P 77d2dd07d67e4b3496a2860f4c88d944: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:18.337707 12485 raft_consensus.cc:348] T b4113be2795944f59de0846cdec3002b P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:18.337797 12485 raft_consensus.cc:374] T b4113be2795944f59de0846cdec3002b P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:18.337817 12485 raft_consensus.cc:729] T b4113be2795944f59de0846cdec3002b P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 433eed7b49a44911bd5764d0f5a97bcc, State: Initialized, Role: FOLLOWER
I20260929 19:10:18.337735 12484 raft_consensus.cc:348] T b4113be2795944f59de0846cdec3002b P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:18.337877 12484 raft_consensus.cc:374] T b4113be2795944f59de0846cdec3002b P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:18.337898 12484 raft_consensus.cc:729] T b4113be2795944f59de0846cdec3002b P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77d2dd07d67e4b3496a2860f4c88d944, State: Initialized, Role: FOLLOWER
I20260929 19:10:18.337898 12485 consensus_queue.cc:260] T b4113be2795944f59de0846cdec3002b P 433eed7b49a44911bd5764d0f5a97bcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:18.337953 12484 consensus_queue.cc:260] T b4113be2795944f59de0846cdec3002b P 77d2dd07d67e4b3496a2860f4c88d944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:18.338078 12485 ts_tablet_manager.cc:1428] T b4113be2795944f59de0846cdec3002b P 433eed7b49a44911bd5764d0f5a97bcc: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260929 19:10:18.338124 12484 ts_tablet_manager.cc:1428] T b4113be2795944f59de0846cdec3002b P 77d2dd07d67e4b3496a2860f4c88d944: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:18.338573 12486 tablet_bootstrap.cc:492] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51: No bootstrap required, opened a new log
I20260929 19:10:18.338627 12486 ts_tablet_manager.cc:1397] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:18.338888 12486 raft_consensus.cc:348] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:18.338970 12486 raft_consensus.cc:374] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:18.338990 12486 raft_consensus.cc:729] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6500e0146cf84a229ff00c3d1e1c7f51, State: Initialized, Role: FOLLOWER
I20260929 19:10:18.339031 12486 consensus_queue.cc:260] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:18.339128 12486 ts_tablet_manager.cc:1428] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:18.407509 12454 raft_consensus.cc:482] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:10:18.407656 12454 raft_consensus.cc:504] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:18.407877 12454 leader_election.cc:288] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 433eed7b49a44911bd5764d0f5a97bcc (127.9.236.2:37111), 77d2dd07d67e4b3496a2860f4c88d944 (127.9.236.3:35771)
I20260929 19:10:18.408087 12185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4113be2795944f59de0846cdec3002b" candidate_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "433eed7b49a44911bd5764d0f5a97bcc" is_pre_election: true
I20260929 19:10:18.408094 12322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4113be2795944f59de0846cdec3002b" candidate_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77d2dd07d67e4b3496a2860f4c88d944" is_pre_election: true
I20260929 19:10:18.408202 12185 raft_consensus.cc:2454] T b4113be2795944f59de0846cdec3002b P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6500e0146cf84a229ff00c3d1e1c7f51 in term 0.
I20260929 19:10:18.408228 12322 raft_consensus.cc:2454] T b4113be2795944f59de0846cdec3002b P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6500e0146cf84a229ff00c3d1e1c7f51 in term 0.
I20260929 19:10:18.408406 11978 leader_election.cc:302] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 433eed7b49a44911bd5764d0f5a97bcc, 6500e0146cf84a229ff00c3d1e1c7f51; no voters:
I20260929 19:10:18.408552 12454 raft_consensus.cc:2781] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:10:18.408600 12454 raft_consensus.cc:482] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:10:18.408618 12454 raft_consensus.cc:3037] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:18.409286 12454 raft_consensus.cc:504] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:18.409415 12454 leader_election.cc:288] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 [CANDIDATE]: Term 1 election: Requested vote from peers 433eed7b49a44911bd5764d0f5a97bcc (127.9.236.2:37111), 77d2dd07d67e4b3496a2860f4c88d944 (127.9.236.3:35771)
I20260929 19:10:18.409601 12185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4113be2795944f59de0846cdec3002b" candidate_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "433eed7b49a44911bd5764d0f5a97bcc"
I20260929 19:10:18.409591 12322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4113be2795944f59de0846cdec3002b" candidate_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77d2dd07d67e4b3496a2860f4c88d944"
I20260929 19:10:18.409672 12185 raft_consensus.cc:3037] T b4113be2795944f59de0846cdec3002b P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:18.409672 12322 raft_consensus.cc:3037] T b4113be2795944f59de0846cdec3002b P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:18.410593 12185 raft_consensus.cc:2454] T b4113be2795944f59de0846cdec3002b P 433eed7b49a44911bd5764d0f5a97bcc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6500e0146cf84a229ff00c3d1e1c7f51 in term 1.
I20260929 19:10:18.410593 12322 raft_consensus.cc:2454] T b4113be2795944f59de0846cdec3002b P 77d2dd07d67e4b3496a2860f4c88d944 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6500e0146cf84a229ff00c3d1e1c7f51 in term 1.
I20260929 19:10:18.410779 11979 leader_election.cc:302] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6500e0146cf84a229ff00c3d1e1c7f51, 77d2dd07d67e4b3496a2860f4c88d944; no voters:
I20260929 19:10:18.410903 12454 raft_consensus.cc:2781] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:18.410959 12454 raft_consensus.cc:686] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 [term 1 LEADER]: Becoming Leader. State: Replica: 6500e0146cf84a229ff00c3d1e1c7f51, State: Running, Role: LEADER
I20260929 19:10:18.411000 12454 consensus_queue.cc:237] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:18.411619 11887 catalog_manager.cc:5495] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6500e0146cf84a229ff00c3d1e1c7f51 (127.9.236.1). New cstate: current_term: 1 leader_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } health_report { overall_health: UNKNOWN } } }
I20260929 19:10:18.433869 12493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:18.431328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57392 (local address 127.9.236.1:33007)
0929 19:10:18.431532 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:18.431534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:18.431543 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:18.431564 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:18.431565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:18.431579 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:18.431653 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:18.431827 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:18.432351 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:18.433094 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:18.433358 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:18.433433 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:18.433567 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:18.433600 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:18.433647 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:18.433704 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 19:10:18.433727 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":132,"threads_started":1}
I20260929 19:10:18.438517 12493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:18.436330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:40387 (local address 127.9.236.1:33007)
0929 19:10:18.436409 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:18.436411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:18.436424 (+ 13us) server_negotiation.cc:407] Connection header received
0929 19:10:18.436493 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:18.436494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:18.436505 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:18.436559 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:18.436668 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:18.437155 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:18.437894 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:18.438082 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:18.438143 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:18.438255 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:18.438294 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:18.438339 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:18.438401 (+ 62us) server_negotiation.cc:299] Negotiation successful
0929 19:10:18.438423 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 19:10:18.438529 12496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:18.436226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:33007 (local address 127.9.236.1:40387)
0929 19:10:18.436362 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:18.436376 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:18.436438 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:18.436566 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:18.436567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:18.436569 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:10:18.436614 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:18.436615 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:18.437238 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:18.437239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:18.437816 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:18.437818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:18.438145 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:18.438146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:18.438197 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:18.438200 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:10:18.438344 (+ 144us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:10:18.438346 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:10:18.438398 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 19:10:18.438446 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":1,"thread_start_us":68,"threads_started":1}
I20260929 19:10:18.441017 12322 raft_consensus.cc:1261] T b4113be2795944f59de0846cdec3002b P 77d2dd07d67e4b3496a2860f4c88d944 [term 1 FOLLOWER]: Refusing update from remote peer 6500e0146cf84a229ff00c3d1e1c7f51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:18.441048 12185 raft_consensus.cc:1261] T b4113be2795944f59de0846cdec3002b P 433eed7b49a44911bd5764d0f5a97bcc [term 1 FOLLOWER]: Refusing update from remote peer 6500e0146cf84a229ff00c3d1e1c7f51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:18.441861 12459 consensus_queue.cc:1035] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:18.442006 12451 consensus_queue.cc:1035] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:18.453145 11887 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:18.453277 11887 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:18.454169 11887 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417: Sending DeleteTablet for 3 replicas of tablet b4113be2795944f59de0846cdec3002b
I20260929 19:10:18.454399 12024 tablet_service.cc:1514] Processing DeleteTablet for tablet b4113be2795944f59de0846cdec3002b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:18 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:57370
I20260929 19:10:18.454519 12305 tablet_service.cc:1514] Processing DeleteTablet for tablet b4113be2795944f59de0846cdec3002b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:18 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:48524
I20260929 19:10:18.454533 12165 tablet_service.cc:1514] Processing DeleteTablet for tablet b4113be2795944f59de0846cdec3002b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:18 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:51364
I20260929 19:10:18.454631 12473 tablet_replica.cc:330] T b4113be2795944f59de0846cdec3002b P 433eed7b49a44911bd5764d0f5a97bcc: stopping tablet replica
I20260929 19:10:18.454638 12474 tablet_replica.cc:330] T b4113be2795944f59de0846cdec3002b P 77d2dd07d67e4b3496a2860f4c88d944: stopping tablet replica
I20260929 19:10:18.454641 12475 tablet_replica.cc:330] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51: stopping tablet replica
I20260929 19:10:18.454684 12473 raft_consensus.cc:2229] T b4113be2795944f59de0846cdec3002b P 433eed7b49a44911bd5764d0f5a97bcc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:18.454694 12474 raft_consensus.cc:2229] T b4113be2795944f59de0846cdec3002b P 77d2dd07d67e4b3496a2860f4c88d944 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:18.454701 12475 raft_consensus.cc:2229] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:10:18.454721 12473 raft_consensus.cc:2258] T b4113be2795944f59de0846cdec3002b P 433eed7b49a44911bd5764d0f5a97bcc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:18.454741 12474 raft_consensus.cc:2258] T b4113be2795944f59de0846cdec3002b P 77d2dd07d67e4b3496a2860f4c88d944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:18.454793 12475 raft_consensus.cc:2258] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:18.455132 12475 ts_tablet_manager.cc:1905] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:18.455077 11887 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:10:18.455216 11887 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:10:18.455495 12474 ts_tablet_manager.cc:1905] T b4113be2795944f59de0846cdec3002b P 77d2dd07d67e4b3496a2860f4c88d944: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:18.456261 12024 tablet_service.cc:1467] Processing CreateTablet for tablet 928cad3b9f5c4c9895b744e5c8b6c050 (DEFAULT_TABLE table=test-table [id=3fa25942140e4375b2d135219ecba77d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:18.456290 12165 tablet_service.cc:1467] Processing CreateTablet for tablet 928cad3b9f5c4c9895b744e5c8b6c050 (DEFAULT_TABLE table=test-table [id=3fa25942140e4375b2d135219ecba77d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:18.456370 12024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 928cad3b9f5c4c9895b744e5c8b6c050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:18.456424 12165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 928cad3b9f5c4c9895b744e5c8b6c050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:18.457168 12474 ts_tablet_manager.cc:1918] T b4113be2795944f59de0846cdec3002b P 77d2dd07d67e4b3496a2860f4c88d944: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:10:18.457182 12475 ts_tablet_manager.cc:1918] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:10:18.457226 12474 log.cc:1196] T b4113be2795944f59de0846cdec3002b P 77d2dd07d67e4b3496a2860f4c88d944: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/b4113be2795944f59de0846cdec3002b
I20260929 19:10:18.457232 12475 log.cc:1196] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-0/wal/wals/b4113be2795944f59de0846cdec3002b
I20260929 19:10:18.457484 12474 ts_tablet_manager.cc:1939] T b4113be2795944f59de0846cdec3002b P 77d2dd07d67e4b3496a2860f4c88d944: Deleting consensus metadata
I20260929 19:10:18.457530 12475 ts_tablet_manager.cc:1939] T b4113be2795944f59de0846cdec3002b P 6500e0146cf84a229ff00c3d1e1c7f51: Deleting consensus metadata
I20260929 19:10:18.457979 12485 tablet_bootstrap.cc:492] T 928cad3b9f5c4c9895b744e5c8b6c050 P 433eed7b49a44911bd5764d0f5a97bcc: Bootstrap starting.
I20260929 19:10:18.458070 12486 tablet_bootstrap.cc:492] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51: Bootstrap starting.
I20260929 19:10:18.458189 11873 catalog_manager.cc:4841] TS 6500e0146cf84a229ff00c3d1e1c7f51 (127.9.236.1:33007): tablet b4113be2795944f59de0846cdec3002b (table test-table [id=8cce6c2a2f534a74b06c8c4847cd4b48]) successfully deleted
I20260929 19:10:18.458230 11875 catalog_manager.cc:4841] TS 77d2dd07d67e4b3496a2860f4c88d944 (127.9.236.3:35771): tablet b4113be2795944f59de0846cdec3002b (table test-table [id=8cce6c2a2f534a74b06c8c4847cd4b48]) successfully deleted
I20260929 19:10:18.458390 12486 tablet_bootstrap.cc:654] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:18.458436 12485 tablet_bootstrap.cc:654] T 928cad3b9f5c4c9895b744e5c8b6c050 P 433eed7b49a44911bd5764d0f5a97bcc: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:18.458686 12305 tablet_service.cc:1467] Processing CreateTablet for tablet 928cad3b9f5c4c9895b744e5c8b6c050 (DEFAULT_TABLE table=test-table [id=3fa25942140e4375b2d135219ecba77d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:18.458793 12305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 928cad3b9f5c4c9895b744e5c8b6c050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:18.459306 12485 tablet_bootstrap.cc:492] T 928cad3b9f5c4c9895b744e5c8b6c050 P 433eed7b49a44911bd5764d0f5a97bcc: No bootstrap required, opened a new log
I20260929 19:10:18.459362 12485 ts_tablet_manager.cc:1397] T 928cad3b9f5c4c9895b744e5c8b6c050 P 433eed7b49a44911bd5764d0f5a97bcc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:18.459502 12485 raft_consensus.cc:348] T 928cad3b9f5c4c9895b744e5c8b6c050 P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:18.459579 12485 raft_consensus.cc:374] T 928cad3b9f5c4c9895b744e5c8b6c050 P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:18.459599 12485 raft_consensus.cc:729] T 928cad3b9f5c4c9895b744e5c8b6c050 P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 433eed7b49a44911bd5764d0f5a97bcc, State: Initialized, Role: FOLLOWER
I20260929 19:10:18.459650 12485 consensus_queue.cc:260] T 928cad3b9f5c4c9895b744e5c8b6c050 P 433eed7b49a44911bd5764d0f5a97bcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:18.459749 12485 ts_tablet_manager.cc:1428] T 928cad3b9f5c4c9895b744e5c8b6c050 P 433eed7b49a44911bd5764d0f5a97bcc: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260929 19:10:18.460366 12484 tablet_bootstrap.cc:492] T 928cad3b9f5c4c9895b744e5c8b6c050 P 77d2dd07d67e4b3496a2860f4c88d944: Bootstrap starting.
I20260929 19:10:18.460799 12484 tablet_bootstrap.cc:654] T 928cad3b9f5c4c9895b744e5c8b6c050 P 77d2dd07d67e4b3496a2860f4c88d944: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:18.461227 12473 ts_tablet_manager.cc:1905] T b4113be2795944f59de0846cdec3002b P 433eed7b49a44911bd5764d0f5a97bcc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:18.462903 12473 ts_tablet_manager.cc:1918] T b4113be2795944f59de0846cdec3002b P 433eed7b49a44911bd5764d0f5a97bcc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:10:18.462913 12486 tablet_bootstrap.cc:492] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51: No bootstrap required, opened a new log
I20260929 19:10:18.462949 12473 log.cc:1196] T b4113be2795944f59de0846cdec3002b P 433eed7b49a44911bd5764d0f5a97bcc: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/b4113be2795944f59de0846cdec3002b
I20260929 19:10:18.462960 12486 ts_tablet_manager.cc:1397] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260929 19:10:18.463124 12473 ts_tablet_manager.cc:1939] T b4113be2795944f59de0846cdec3002b P 433eed7b49a44911bd5764d0f5a97bcc: Deleting consensus metadata
I20260929 19:10:18.463083 12486 raft_consensus.cc:348] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:18.463148 12486 raft_consensus.cc:374] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:18.463171 12486 raft_consensus.cc:729] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6500e0146cf84a229ff00c3d1e1c7f51, State: Initialized, Role: FOLLOWER
I20260929 19:10:18.463217 12486 consensus_queue.cc:260] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:18.463304 12486 ts_tablet_manager.cc:1428] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:18.463423 11874 catalog_manager.cc:4841] TS 433eed7b49a44911bd5764d0f5a97bcc (127.9.236.2:37111): tablet b4113be2795944f59de0846cdec3002b (table test-table [id=8cce6c2a2f534a74b06c8c4847cd4b48]) successfully deleted
I20260929 19:10:18.463944 12484 tablet_bootstrap.cc:492] T 928cad3b9f5c4c9895b744e5c8b6c050 P 77d2dd07d67e4b3496a2860f4c88d944: No bootstrap required, opened a new log
I20260929 19:10:18.464005 12484 ts_tablet_manager.cc:1397] T 928cad3b9f5c4c9895b744e5c8b6c050 P 77d2dd07d67e4b3496a2860f4c88d944: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260929 19:10:18.464186 12484 raft_consensus.cc:348] T 928cad3b9f5c4c9895b744e5c8b6c050 P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:18.464253 12484 raft_consensus.cc:374] T 928cad3b9f5c4c9895b744e5c8b6c050 P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:18.464267 12484 raft_consensus.cc:729] T 928cad3b9f5c4c9895b744e5c8b6c050 P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77d2dd07d67e4b3496a2860f4c88d944, State: Initialized, Role: FOLLOWER
I20260929 19:10:18.464324 12484 consensus_queue.cc:260] T 928cad3b9f5c4c9895b744e5c8b6c050 P 77d2dd07d67e4b3496a2860f4c88d944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:18.464470 12484 ts_tablet_manager.cc:1428] T 928cad3b9f5c4c9895b744e5c8b6c050 P 77d2dd07d67e4b3496a2860f4c88d944: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260929 19:10:18.629644 12454 raft_consensus.cc:482] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:10:18.629761 12454 raft_consensus.cc:504] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:18.629985 12454 leader_election.cc:288] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 433eed7b49a44911bd5764d0f5a97bcc (127.9.236.2:37111), 77d2dd07d67e4b3496a2860f4c88d944 (127.9.236.3:35771)
I20260929 19:10:18.630270 12185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "928cad3b9f5c4c9895b744e5c8b6c050" candidate_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "433eed7b49a44911bd5764d0f5a97bcc" is_pre_election: true
I20260929 19:10:18.630373 12185 raft_consensus.cc:2454] T 928cad3b9f5c4c9895b744e5c8b6c050 P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6500e0146cf84a229ff00c3d1e1c7f51 in term 0.
I20260929 19:10:18.630290 12322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "928cad3b9f5c4c9895b744e5c8b6c050" candidate_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77d2dd07d67e4b3496a2860f4c88d944" is_pre_election: true
I20260929 19:10:18.630414 12322 raft_consensus.cc:2454] T 928cad3b9f5c4c9895b744e5c8b6c050 P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6500e0146cf84a229ff00c3d1e1c7f51 in term 0.
I20260929 19:10:18.630582 11978 leader_election.cc:302] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 433eed7b49a44911bd5764d0f5a97bcc, 6500e0146cf84a229ff00c3d1e1c7f51; no voters:
I20260929 19:10:18.630707 12454 raft_consensus.cc:2781] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:10:18.630769 12454 raft_consensus.cc:482] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:10:18.630789 12454 raft_consensus.cc:3037] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:18.631464 12454 raft_consensus.cc:504] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:18.631611 12454 leader_election.cc:288] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51 [CANDIDATE]: Term 1 election: Requested vote from peers 433eed7b49a44911bd5764d0f5a97bcc (127.9.236.2:37111), 77d2dd07d67e4b3496a2860f4c88d944 (127.9.236.3:35771)
I20260929 19:10:18.631754 12322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "928cad3b9f5c4c9895b744e5c8b6c050" candidate_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77d2dd07d67e4b3496a2860f4c88d944"
I20260929 19:10:18.631749 12185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "928cad3b9f5c4c9895b744e5c8b6c050" candidate_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "433eed7b49a44911bd5764d0f5a97bcc"
I20260929 19:10:18.631824 12185 raft_consensus.cc:3037] T 928cad3b9f5c4c9895b744e5c8b6c050 P 433eed7b49a44911bd5764d0f5a97bcc [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:18.631819 12322 raft_consensus.cc:3037] T 928cad3b9f5c4c9895b744e5c8b6c050 P 77d2dd07d67e4b3496a2860f4c88d944 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:18.632408 12322 raft_consensus.cc:2454] T 928cad3b9f5c4c9895b744e5c8b6c050 P 77d2dd07d67e4b3496a2860f4c88d944 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6500e0146cf84a229ff00c3d1e1c7f51 in term 1.
I20260929 19:10:18.632557 11979 leader_election.cc:302] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6500e0146cf84a229ff00c3d1e1c7f51, 77d2dd07d67e4b3496a2860f4c88d944; no voters:
I20260929 19:10:18.632647 12185 raft_consensus.cc:2454] T 928cad3b9f5c4c9895b744e5c8b6c050 P 433eed7b49a44911bd5764d0f5a97bcc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6500e0146cf84a229ff00c3d1e1c7f51 in term 1.
I20260929 19:10:18.632659 12454 raft_consensus.cc:2781] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:18.632721 12454 raft_consensus.cc:686] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51 [term 1 LEADER]: Becoming Leader. State: Replica: 6500e0146cf84a229ff00c3d1e1c7f51, State: Running, Role: LEADER
I20260929 19:10:18.632762 12454 consensus_queue.cc:237] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } }
I20260929 19:10:18.633402 11887 catalog_manager.cc:5495] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6500e0146cf84a229ff00c3d1e1c7f51 (127.9.236.1). New cstate: current_term: 1 leader_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6500e0146cf84a229ff00c3d1e1c7f51" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 33007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 } health_report { overall_health: UNKNOWN } } }
I20260929 19:10:18.644194 12322 raft_consensus.cc:1261] T 928cad3b9f5c4c9895b744e5c8b6c050 P 77d2dd07d67e4b3496a2860f4c88d944 [term 1 FOLLOWER]: Refusing update from remote peer 6500e0146cf84a229ff00c3d1e1c7f51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:18.644225 12185 raft_consensus.cc:1261] T 928cad3b9f5c4c9895b744e5c8b6c050 P 433eed7b49a44911bd5764d0f5a97bcc [term 1 FOLLOWER]: Refusing update from remote peer 6500e0146cf84a229ff00c3d1e1c7f51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:18.644421 12451 consensus_queue.cc:1035] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77d2dd07d67e4b3496a2860f4c88d944" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:18.644526 12459 consensus_queue.cc:1035] T 928cad3b9f5c4c9895b744e5c8b6c050 P 6500e0146cf84a229ff00c3d1e1c7f51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "433eed7b49a44911bd5764d0f5a97bcc" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 19:10:18.649369 12452 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 19:10:20.168889 11887 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:20.169000 11887 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:20.169898 11887 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 492fa2d090474b6a83e5d270db635417: Sending DeleteTablet for 3 replicas of tablet 928cad3b9f5c4c9895b744e5c8b6c050
I20260929 19:10:20.170142 12024 tablet_service.cc:1514] Processing DeleteTablet for tablet 928cad3b9f5c4c9895b744e5c8b6c050 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:20 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:57370
I20260929 19:10:20.170192 12165 tablet_service.cc:1514] Processing DeleteTablet for tablet 928cad3b9f5c4c9895b744e5c8b6c050 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:20 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:51364
I20260929 19:10:20.170373 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 11960
I20260929 19:10:20.170470 12305 tablet_service.cc:1514] Processing DeleteTablet for tablet 928cad3b9f5c4c9895b744e5c8b6c050 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:20 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:48524
I20260929 19:10:20.171653 12527 tablet_replica.cc:330] T 928cad3b9f5c4c9895b744e5c8b6c050 P 433eed7b49a44911bd5764d0f5a97bcc: stopping tablet replica
I20260929 19:10:20.171731 12527 raft_consensus.cc:2229] T 928cad3b9f5c4c9895b744e5c8b6c050 P 433eed7b49a44911bd5764d0f5a97bcc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:20.171780 12527 raft_consensus.cc:2258] T 928cad3b9f5c4c9895b744e5c8b6c050 P 433eed7b49a44911bd5764d0f5a97bcc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:20.172063 12527 ts_tablet_manager.cc:1905] T 928cad3b9f5c4c9895b744e5c8b6c050 P 433eed7b49a44911bd5764d0f5a97bcc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:20.173244 12528 tablet_replica.cc:330] T 928cad3b9f5c4c9895b744e5c8b6c050 P 77d2dd07d67e4b3496a2860f4c88d944: stopping tablet replica
I20260929 19:10:20.173353 12528 raft_consensus.cc:2229] T 928cad3b9f5c4c9895b744e5c8b6c050 P 77d2dd07d67e4b3496a2860f4c88d944 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:20.173405 12528 raft_consensus.cc:2258] T 928cad3b9f5c4c9895b744e5c8b6c050 P 77d2dd07d67e4b3496a2860f4c88d944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:20.173651 12528 ts_tablet_manager.cc:1905] T 928cad3b9f5c4c9895b744e5c8b6c050 P 77d2dd07d67e4b3496a2860f4c88d944: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:20.174124 12527 ts_tablet_manager.cc:1918] T 928cad3b9f5c4c9895b744e5c8b6c050 P 433eed7b49a44911bd5764d0f5a97bcc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 19:10:20.174186 12527 log.cc:1196] T 928cad3b9f5c4c9895b744e5c8b6c050 P 433eed7b49a44911bd5764d0f5a97bcc: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/928cad3b9f5c4c9895b744e5c8b6c050
I20260929 19:10:20.174403 12527 ts_tablet_manager.cc:1939] T 928cad3b9f5c4c9895b744e5c8b6c050 P 433eed7b49a44911bd5764d0f5a97bcc: Deleting consensus metadata
I20260929 19:10:20.174772 11874 catalog_manager.cc:4841] TS 433eed7b49a44911bd5764d0f5a97bcc (127.9.236.2:37111): tablet 928cad3b9f5c4c9895b744e5c8b6c050 (table test-table [id=3fa25942140e4375b2d135219ecba77d]) successfully deleted
I20260929 19:10:20.175191 12528 ts_tablet_manager.cc:1918] T 928cad3b9f5c4c9895b744e5c8b6c050 P 77d2dd07d67e4b3496a2860f4c88d944: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 19:10:20.175244 12528 log.cc:1196] T 928cad3b9f5c4c9895b744e5c8b6c050 P 77d2dd07d67e4b3496a2860f4c88d944: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/928cad3b9f5c4c9895b744e5c8b6c050
I20260929 19:10:20.175436 12528 ts_tablet_manager.cc:1939] T 928cad3b9f5c4c9895b744e5c8b6c050 P 77d2dd07d67e4b3496a2860f4c88d944: Deleting consensus metadata
I20260929 19:10:20.175872 11875 catalog_manager.cc:4841] TS 77d2dd07d67e4b3496a2860f4c88d944 (127.9.236.3:35771): tablet 928cad3b9f5c4c9895b744e5c8b6c050 (table test-table [id=3fa25942140e4375b2d135219ecba77d]) successfully deleted
W20260929 19:10:20.177134 11873 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.236.1:33007 (error 108)
I20260929 19:10:20.177359 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 12100
I20260929 19:10:20.183344 12530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:20.177474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:33007 (local address 127.0.0.1:57406)
0929 19:10:20.183209 (+ 5735us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:20.183282 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.1:33007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5697,"thread_start_us":65,"threads_started":1}
W20260929 19:10:20.183491 11873 catalog_manager.cc:4584] TS 6500e0146cf84a229ff00c3d1e1c7f51 (127.9.236.1:33007): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 928cad3b9f5c4c9895b744e5c8b6c050: Network error: Client connection negotiation failed: client connection to 127.9.236.1:33007: connect: Connection refused (error 111)
I20260929 19:10:20.183480 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 12240
I20260929 19:10:20.187992 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 11856
2026-09-29T19:10:20Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5648 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:10:20 dist-test-slave-wftq krb5kdc[12533](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:10:20 dist-test-slave-wftq krb5kdc[12533](info): set up 2 sockets
Sep 29 19:10:20 dist-test-slave-wftq krb5kdc[12533](info): commencing operation
krb5kdc: starting...
W20260929 19:10:21.140831 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:10:21 dist-test-slave-wftq krb5kdc[12533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:10:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:10:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:10:21.262104 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:33097
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:33397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:33097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:21.347150 12546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:21.347352 12546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:21.347374 12546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:21.348520 12546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:21.348563 12546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:21.348577 12546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:21.348587 12546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:21.348598 12546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:21.350107 12546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:33397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:33097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:33097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:21.350327 12546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:21.350551 12546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:21.355633 12551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:21.355656 12554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:21.355762 12546 server_base.cc:1034] running on GCE node
W20260929 19:10:21.355685 12552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:21.356221 12546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:21.356518 12546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:21.357667 12546 hybrid_clock.cc:648] HybridClock initialized: now 1790709021357642 us; error 39 us; skew 500 ppm
Sep 29 19:10:21 dist-test-slave-wftq krb5kdc[12533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709021, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:21.360031 12546 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:10:21.360441 12546 webserver.cc:460] Webserver started at http://127.9.236.62:41433/ using document root <none> and password file <none>
I20260929 19:10:21.360643 12546 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:21.360685 12546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:21.360769 12546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:21.361760 12546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "c68f5c7fc873431b9fb6a58c96483ab2"
format_stamp: "Formatted at 2026-09-29 19:10:21 on dist-test-slave-wftq"
server_key: "0d1570d523b8f343d5687687853370aa"
server_key_iv: "ebedbdcd25a270e1873cc927f1049a22"
server_key_version: "encryptionkey@0"
I20260929 19:10:21.362046 12546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "c68f5c7fc873431b9fb6a58c96483ab2"
format_stamp: "Formatted at 2026-09-29 19:10:21 on dist-test-slave-wftq"
server_key: "0d1570d523b8f343d5687687853370aa"
server_key_iv: "ebedbdcd25a270e1873cc927f1049a22"
server_key_version: "encryptionkey@0"
I20260929 19:10:21.363233 12546 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:10:21.363920 12561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:21.364115 12546 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:21.364208 12546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "c68f5c7fc873431b9fb6a58c96483ab2"
format_stamp: "Formatted at 2026-09-29 19:10:21 on dist-test-slave-wftq"
server_key: "0d1570d523b8f343d5687687853370aa"
server_key_iv: "ebedbdcd25a270e1873cc927f1049a22"
server_key_version: "encryptionkey@0"
I20260929 19:10:21.364285 12546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:21.378350 12546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:21.380115 12546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:21.380226 12546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:21.384650 12546 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:33097
I20260929 19:10:21.384732 12613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:33097 every 8 connection(s)
I20260929 19:10:21.385154 12546 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:10:21.386089 12614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:21.386350 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 12546
I20260929 19:10:21.386435 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:10:21.386631 10160 external_mini_cluster.cc:1428] Setting key 273f5aff0992d969ff425cadaf195a80
I20260929 19:10:21.388939 12614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2: Bootstrap starting.
I20260929 19:10:21.389562 12614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:21.389822 12614 log.cc:824] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:21.390450 12614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2: No bootstrap required, opened a new log
I20260929 19:10:21.392659 12614 raft_consensus.cc:348] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c68f5c7fc873431b9fb6a58c96483ab2" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 33097 } }
I20260929 19:10:21.392838 12614 raft_consensus.cc:374] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:21.392874 12614 raft_consensus.cc:729] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c68f5c7fc873431b9fb6a58c96483ab2, State: Initialized, Role: FOLLOWER
I20260929 19:10:21.393015 12614 consensus_queue.cc:260] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c68f5c7fc873431b9fb6a58c96483ab2" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 33097 } }
I20260929 19:10:21.393131 12614 raft_consensus.cc:388] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:10:21.393177 12614 raft_consensus.cc:482] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:10:21.393229 12614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2 [term 0 FOLLOWER]: Advancing to term 1
Sep 29 19:10:21 dist-test-slave-wftq krb5kdc[12533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:21.393831 12614 raft_consensus.cc:504] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c68f5c7fc873431b9fb6a58c96483ab2" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 33097 } }
I20260929 19:10:21.393939 12614 leader_election.cc:302] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c68f5c7fc873431b9fb6a58c96483ab2; no voters:
I20260929 19:10:21.394119 12614 leader_election.cc:288] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:10:21.394196 12619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:21.394395 12614 sys_catalog.cc:564] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:21.394398 12619 raft_consensus.cc:686] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2 [term 1 LEADER]: Becoming Leader. State: Replica: c68f5c7fc873431b9fb6a58c96483ab2, State: Running, Role: LEADER
I20260929 19:10:21.394984 12619 consensus_queue.cc:237] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c68f5c7fc873431b9fb6a58c96483ab2" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 33097 } }
I20260929 19:10:21.396198 12621 sys_catalog.cc:455] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c68f5c7fc873431b9fb6a58c96483ab2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c68f5c7fc873431b9fb6a58c96483ab2" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 33097 } } }
I20260929 19:10:21.396503 12621 sys_catalog.cc:458] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:21.396504 12620 sys_catalog.cc:455] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c68f5c7fc873431b9fb6a58c96483ab2. Latest consensus state: current_term: 1 leader_uuid: "c68f5c7fc873431b9fb6a58c96483ab2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c68f5c7fc873431b9fb6a58c96483ab2" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 33097 } } }
I20260929 19:10:21.396602 12620 sys_catalog.cc:458] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2 [sys.catalog]: This master's current role is: LEADER
W20260929 19:10:21.396800 12634 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:10:21.396875 12634 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:10:21.397867 12617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:21.387151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43718 (local address 127.9.236.62:33097)
0929 19:10:21.387336 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:21.387344 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:21.387375 (+ 31us) server_negotiation.cc:407] Connection header received
0929 19:10:21.387889 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:21.387897 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:21.387945 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:21.388073 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:21.388827 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:21.389852 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:21.390651 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:21.391277 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:21.391409 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:21.393813 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:21.393817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:21.393817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:21.393831 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:21.396028 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:21.396445 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:21.396447 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:21.396448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:21.396561 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:21.396806 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:21.396808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:21.396808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:21.397200 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:21.397367 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:21.397494 (+ 127us) server_negotiation.cc:299] Negotiation successful
0929 19:10:21.397655 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":67,"threads_started":1}
I20260929 19:10:21.398200 12636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:21.398464 12636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:10:21.400285 12636 catalog_manager.cc:1347] Generated new cluster ID: 9d214bdcafab4624b219ce4a6820588c
I20260929 19:10:21.400352 12636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:10:21.448168 12636 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:10:21.449229 12636 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:21.451956 12636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c68f5c7fc873431b9fb6a58c96483ab2: Generated new TSK 0
I20260929 19:10:21.452111 12636 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.236.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
I20260929 19:10:21.484130 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:33097
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:33397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:21.568441 12640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:21.568638 12640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:21.568660 12640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:21.569841 12640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:21.569886 12640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:21.569911 12640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:10:21.571244 12640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:33397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:33097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:21.571417 12640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:21.571595 12640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:21.576656 12646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:21.576690 12648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:21.576675 12645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:21.576805 12640 server_base.cc:1034] running on GCE node
I20260929 19:10:21.577252 12640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:21.577515 12640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:21.578655 12640 hybrid_clock.cc:648] HybridClock initialized: now 1790709021578631 us; error 46 us; skew 500 ppm
Sep 29 19:10:21 dist-test-slave-wftq krb5kdc[12533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709021, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:21.581280 12640 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:10:21.581754 12640 webserver.cc:460] Webserver started at http://127.9.236.1:41549/ using document root <none> and password file <none>
I20260929 19:10:21.581998 12640 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:21.582049 12640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:21.582183 12640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:21.583377 12640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "c9794dc9c191451c880274ab26b2f3aa"
format_stamp: "Formatted at 2026-09-29 19:10:21 on dist-test-slave-wftq"
server_key: "2cc3c6ffe1d2b02d5494104dc35e61ec"
server_key_iv: "6e12360eeb4135b0593c2022046cb081"
server_key_version: "encryptionkey@0"
I20260929 19:10:21.583724 12640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "c9794dc9c191451c880274ab26b2f3aa"
format_stamp: "Formatted at 2026-09-29 19:10:21 on dist-test-slave-wftq"
server_key: "2cc3c6ffe1d2b02d5494104dc35e61ec"
server_key_iv: "6e12360eeb4135b0593c2022046cb081"
server_key_version: "encryptionkey@0"
I20260929 19:10:21.585180 12640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:10:21.585973 12655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:21.586163 12640 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:21.586248 12640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "c9794dc9c191451c880274ab26b2f3aa"
format_stamp: "Formatted at 2026-09-29 19:10:21 on dist-test-slave-wftq"
server_key: "2cc3c6ffe1d2b02d5494104dc35e61ec"
server_key_iv: "6e12360eeb4135b0593c2022046cb081"
server_key_version: "encryptionkey@0"
I20260929 19:10:21.586323 12640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:21.595436 12640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:21.597506 12640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:21.597646 12640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:21.597862 12640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:21.598157 12640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:21.598189 12640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:21.598217 12640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:21.598233 12640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:21.604734 12640 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:44231
I20260929 19:10:21.604826 12768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:44231 every 8 connection(s)
I20260929 19:10:21.605139 12640 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:10:21.609558 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 12640
I20260929 19:10:21.609642 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:10:21.609845 10160 external_mini_cluster.cc:1428] Setting key 06e9ecd5cbf89a077ebe3a67e9744bc6
Sep 29 19:10:21 dist-test-slave-wftq krb5kdc[12533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709021, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:21.615633 12617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:21.605962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:54093 (local address 127.9.236.62:33097)
0929 19:10:21.606156 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:21.606159 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:21.606250 (+ 91us) server_negotiation.cc:407] Connection header received
0929 19:10:21.606928 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:21.606929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:21.606943 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:21.607015 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:21.608249 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:21.608892 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:21.610148 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:21.610429 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:21.610503 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:21.612846 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:21.612848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:21.612848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:21.612856 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:21.614301 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:21.614726 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:21.614728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:21.614729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:21.614801 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:21.615004 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:21.615005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:21.615006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:21.615125 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:21.615186 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:21.615416 (+ 230us) server_negotiation.cc:299] Negotiation successful
0929 19:10:21.615526 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260929 19:10:21.615739 12771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:21.605906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:33097 (local address 127.9.236.1:54093)
0929 19:10:21.606097 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:21.606146 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:21.606809 (+ 663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:21.607092 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:21.607096 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:21.607825 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:21.608136 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:21.608142 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:21.608975 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:21.608976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:21.610066 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:21.610069 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:21.610519 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:21.610520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:21.610658 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:21.611201 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:21.611208 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:21.612745 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:21.614388 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:21.614394 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:21.614396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:21.614630 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:21.614856 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:21.614857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:21.614857 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:21.614950 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:21.615179 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:21.615185 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:21.615303 (+ 118us) client_negotiation.cc:769] Sending connection context
0929 19:10:21.615388 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 19:10:21.615534 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260929 19:10:21.616150 12769 heartbeater.cc:347] Connected to a master server at 127.9.236.62:33097
I20260929 19:10:21.616242 12769 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:21.616410 12769 heartbeater.cc:511] Master 127.9.236.62:33097 requested a full tablet report, sending...
I20260929 19:10:21.616870 12578 ts_manager.cc:194] Registered new tserver with Master: c9794dc9c191451c880274ab26b2f3aa (127.9.236.1:44231)
I20260929 19:10:21.617506 12578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:54093
WARNING: no policy specified for kudu/127.9.236.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
I20260929 19:10:21.635425 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:33097
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:33397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:21.724345 12774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:21.724591 12774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:21.724614 12774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:21.725847 12774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:21.725904 12774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:21.725932 12774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:10:21.727468 12774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:33397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:33097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:21.727697 12774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:21.727908 12774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:21.733439 12779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:21.733443 12782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:21.733477 12774 server_base.cc:1034] running on GCE node
W20260929 19:10:21.733440 12780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:21.733922 12774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:21.734176 12774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:21.735337 12774 hybrid_clock.cc:648] HybridClock initialized: now 1790709021735313 us; error 41 us; skew 500 ppm
Sep 29 19:10:21 dist-test-slave-wftq krb5kdc[12533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709021, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:21.738016 12774 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:10:21.738497 12774 webserver.cc:460] Webserver started at http://127.9.236.2:35709/ using document root <none> and password file <none>
I20260929 19:10:21.738699 12774 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:21.738744 12774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:21.738848 12774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:21.739960 12774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "9de4602efa444d6bbbe8fd13e07fa37c"
format_stamp: "Formatted at 2026-09-29 19:10:21 on dist-test-slave-wftq"
server_key: "3bcaa424c9cdaaecea54f6570cf0c73d"
server_key_iv: "41e5791d0463237909c31abf3f848a20"
server_key_version: "encryptionkey@0"
I20260929 19:10:21.740273 12774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "9de4602efa444d6bbbe8fd13e07fa37c"
format_stamp: "Formatted at 2026-09-29 19:10:21 on dist-test-slave-wftq"
server_key: "3bcaa424c9cdaaecea54f6570cf0c73d"
server_key_iv: "41e5791d0463237909c31abf3f848a20"
server_key_version: "encryptionkey@0"
I20260929 19:10:21.741705 12774 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:10:21.742452 12789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:21.742666 12774 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:21.742762 12774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "9de4602efa444d6bbbe8fd13e07fa37c"
format_stamp: "Formatted at 2026-09-29 19:10:21 on dist-test-slave-wftq"
server_key: "3bcaa424c9cdaaecea54f6570cf0c73d"
server_key_iv: "41e5791d0463237909c31abf3f848a20"
server_key_version: "encryptionkey@0"
I20260929 19:10:21.742846 12774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:21.754809 12774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:21.756708 12774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:21.756819 12774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:21.756994 12774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:21.757310 12774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:21.757354 12774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:21.757373 12774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:21.757393 12774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:21.763480 12774 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:43047
I20260929 19:10:21.763581 12902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:43047 every 8 connection(s)
I20260929 19:10:21.763837 12774 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
Sep 29 19:10:21 dist-test-slave-wftq krb5kdc[12533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709021, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:21.770936 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 12774
I20260929 19:10:21.771025 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:10:21.771207 10160 external_mini_cluster.cc:1428] Setting key 11e08e0ee3e780c6c07edc7d26daed17
I20260929 19:10:21.773018 12617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:21.764618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:38433 (local address 127.9.236.62:33097)
0929 19:10:21.764716 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:21.764718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:21.764870 (+ 152us) server_negotiation.cc:407] Connection header received
0929 19:10:21.765430 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:21.765431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:21.765443 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:21.765501 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:21.766350 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:21.766899 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:21.767952 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:21.768151 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:21.768208 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:21.770272 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:21.770274 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:21.770274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:21.770282 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:21.771648 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:21.772081 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:21.772082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:21.772082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:21.772149 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:21.772365 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:21.772366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:21.772367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:21.772488 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:21.772547 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:21.772836 (+ 289us) server_negotiation.cc:299] Negotiation successful
0929 19:10:21.772925 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 19:10:21.773288 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:21.764567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:33097 (local address 127.9.236.2:38433)
0929 19:10:21.764747 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:21.764786 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:21.765340 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:21.765560 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:21.765563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:21.766088 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:21.766270 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:21.766274 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:21.766983 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:21.766983 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:21.767886 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:21.767889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:21.768217 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:21.768218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:21.768310 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:21.768726 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:21.768733 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:21.770175 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:21.771738 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:21.771743 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:21.771745 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:21.772003 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:21.772204 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:21.772205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:21.772205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:21.772311 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:21.772541 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:21.772547 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:21.772696 (+ 149us) client_negotiation.cc:769] Sending connection context
0929 19:10:21.772795 (+ 99us) client_negotiation.cc:240] Negotiation successful
0929 19:10:21.772968 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":98,"threads_started":1}
I20260929 19:10:21.773769 12903 heartbeater.cc:347] Connected to a master server at 127.9.236.62:33097
I20260929 19:10:21.773869 12903 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:21.774039 12903 heartbeater.cc:511] Master 127.9.236.62:33097 requested a full tablet report, sending...
I20260929 19:10:21.774405 12578 ts_manager.cc:194] Registered new tserver with Master: 9de4602efa444d6bbbe8fd13e07fa37c (127.9.236.2:43047)
I20260929 19:10:21.774770 12578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:38433
WARNING: no policy specified for kudu/127.9.236.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
I20260929 19:10:21.796294 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:33097
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:33397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:21.881388 12908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:21.881618 12908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:21.881640 12908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:21.882850 12908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:21.882905 12908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:21.882931 12908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:10:21.884461 12908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:33397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:33097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:21.884716 12908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:21.884966 12908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:21.890223 12916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:21.890264 12913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:21.890268 12914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:21.890556 12908 server_base.cc:1034] running on GCE node
I20260929 19:10:21.890758 12908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:21.890990 12908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:21.892132 12908 hybrid_clock.cc:648] HybridClock initialized: now 1790709021892119 us; error 35 us; skew 500 ppm
Sep 29 19:10:21 dist-test-slave-wftq krb5kdc[12533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709021, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:21.894683 12908 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:10:21.895202 12908 webserver.cc:460] Webserver started at http://127.9.236.3:41713/ using document root <none> and password file <none>
I20260929 19:10:21.895416 12908 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:21.895458 12908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:21.895556 12908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:21.896701 12908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "db3b2c37ae8a4dd7be238d4ecf07311f"
format_stamp: "Formatted at 2026-09-29 19:10:21 on dist-test-slave-wftq"
server_key: "510eeec2fe6729eef82c006babc6cc87"
server_key_iv: "7fd9bbeddb76279eb25b8e4c2fa9a278"
server_key_version: "encryptionkey@0"
I20260929 19:10:21.897058 12908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "db3b2c37ae8a4dd7be238d4ecf07311f"
format_stamp: "Formatted at 2026-09-29 19:10:21 on dist-test-slave-wftq"
server_key: "510eeec2fe6729eef82c006babc6cc87"
server_key_iv: "7fd9bbeddb76279eb25b8e4c2fa9a278"
server_key_version: "encryptionkey@0"
I20260929 19:10:21.898370 12908 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:21.899070 12923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:21.899263 12908 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:21.899363 12908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "db3b2c37ae8a4dd7be238d4ecf07311f"
format_stamp: "Formatted at 2026-09-29 19:10:21 on dist-test-slave-wftq"
server_key: "510eeec2fe6729eef82c006babc6cc87"
server_key_iv: "7fd9bbeddb76279eb25b8e4c2fa9a278"
server_key_version: "encryptionkey@0"
I20260929 19:10:21.899428 12908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:21.913342 12908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:21.915290 12908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:21.915433 12908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:21.915643 12908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:21.915946 12908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:21.915978 12908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:21.916009 12908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:21.916034 12908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:21.922801 12908 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:43173
I20260929 19:10:21.922906 13036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:43173 every 8 connection(s)
I20260929 19:10:21.923178 12908 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
Sep 29 19:10:21 dist-test-slave-wftq krb5kdc[12533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709021, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:21.931402 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 12908
I20260929 19:10:21.931493 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:10:21.931672 10160 external_mini_cluster.cc:1428] Setting key 7b24c4e8d44d03c4d2062a4181ece6ad
I20260929 19:10:21.932737 12617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:21.923951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:54477 (local address 127.9.236.62:33097)
0929 19:10:21.924040 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:21.924043 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:21.924221 (+ 178us) server_negotiation.cc:407] Connection header received
0929 19:10:21.924771 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:21.924772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:21.924784 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:21.924836 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:21.925861 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:21.926365 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:21.927324 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:21.927519 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:21.927572 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:21.929819 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:21.929820 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:21.929820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:21.929827 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:21.931137 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:21.931610 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:21.931611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:21.931611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:21.931669 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:21.931967 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:21.931968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:21.931968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:21.932104 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:21.932166 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:21.932559 (+ 393us) server_negotiation.cc:299] Negotiation successful
0929 19:10:21.932633 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 19:10:21.932989 13039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:21.923906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:33097 (local address 127.9.236.3:54477)
0929 19:10:21.924096 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:21.924138 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:21.924673 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:21.924903 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:21.924907 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:21.925574 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:21.925776 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:21.925780 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:21.926440 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:21.926441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:21.927248 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:21.927250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:21.927583 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:21.927584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:21.927692 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:21.928159 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:21.928166 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:21.929679 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:21.931250 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:21.931256 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:21.931258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:21.931500 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:21.931780 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:21.931781 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:21.931781 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:21.931904 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:21.932219 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:21.932226 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:21.932396 (+ 170us) client_negotiation.cc:769] Sending connection context
0929 19:10:21.932521 (+ 125us) client_negotiation.cc:240] Negotiation successful
0929 19:10:21.932726 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
I20260929 19:10:21.933550 13037 heartbeater.cc:347] Connected to a master server at 127.9.236.62:33097
I20260929 19:10:21.933650 13037 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:21.933840 13037 heartbeater.cc:511] Master 127.9.236.62:33097 requested a full tablet report, sending...
I20260929 19:10:21.934186 12578 ts_manager.cc:194] Registered new tserver with Master: db3b2c37ae8a4dd7be238d4ecf07311f (127.9.236.3:43173)
I20260929 19:10:21.934517 12578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:54477
I20260929 19:10:21.943564 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 19:10:21.955590 12617 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:21.954617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43726 (local address 127.9.236.62:33097)
0929 19:10:21.954693 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:21.954696 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:21.954800 (+ 104us) server_negotiation.cc:407] Connection header received
0929 19:10:21.954828 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:21.954829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:21.954839 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:21.954915 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:21.955505 (+ 590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43726: BlockingRecv error: recv got EOF from 127.0.0.1:43726 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 19:10:21.956341 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 12640
I20260929 19:10:21.960762 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 12774
I20260929 19:10:21.965169 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 12908
I20260929 19:10:21.969502 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 12546
2026-09-29T19:10:21Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1780 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:10:22 dist-test-slave-wftq krb5kdc[13049](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:10:22 dist-test-slave-wftq krb5kdc[13049](info): set up 2 sockets
Sep 29 19:10:22 dist-test-slave-wftq krb5kdc[13049](info): commencing operation
krb5kdc: starting...
W20260929 19:10:22.940850 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.952s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:10:23 dist-test-slave-wftq krb5kdc[13049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:10:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:10:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:10:23.057878 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:43073
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:40397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:43073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:23.143745 13062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:23.143968 13062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:23.143988 13062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:23.145152 13062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:23.145187 13062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:23.145200 13062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:23.145211 13062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:23.145221 13062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:23.146724 13062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:40397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:43073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:43073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:23.146940 13062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:23.147132 13062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:10:23.152369 13062 server_base.cc:1034] running on GCE node
W20260929 19:10:23.152333 13070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:23.152333 13068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:23.152278 13067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:23.152889 13062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:23.153229 13062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:23.154386 13062 hybrid_clock.cc:648] HybridClock initialized: now 1790709023154369 us; error 35 us; skew 500 ppm
Sep 29 19:10:23 dist-test-slave-wftq krb5kdc[13049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709023, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:23.156891 13062 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:10:23.157358 13062 webserver.cc:460] Webserver started at http://127.9.236.62:44191/ using document root <none> and password file <none>
I20260929 19:10:23.157544 13062 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:23.157588 13062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:23.157678 13062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:23.158785 13062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "6ff55c9f0a4344cdbc6610e15d97e314"
format_stamp: "Formatted at 2026-09-29 19:10:23 on dist-test-slave-wftq"
server_key: "0ce80099e0c71d7722547ec4a1a6ff62"
server_key_iv: "b57ea11d8d86bc40db7a655847cd290b"
server_key_version: "encryptionkey@0"
I20260929 19:10:23.159099 13062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "6ff55c9f0a4344cdbc6610e15d97e314"
format_stamp: "Formatted at 2026-09-29 19:10:23 on dist-test-slave-wftq"
server_key: "0ce80099e0c71d7722547ec4a1a6ff62"
server_key_iv: "b57ea11d8d86bc40db7a655847cd290b"
server_key_version: "encryptionkey@0"
I20260929 19:10:23.160463 13062 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:10:23.161226 13077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:23.161409 13062 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 19:10:23.161511 13062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "6ff55c9f0a4344cdbc6610e15d97e314"
format_stamp: "Formatted at 2026-09-29 19:10:23 on dist-test-slave-wftq"
server_key: "0ce80099e0c71d7722547ec4a1a6ff62"
server_key_iv: "b57ea11d8d86bc40db7a655847cd290b"
server_key_version: "encryptionkey@0"
I20260929 19:10:23.161617 13062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:23.184376 13062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:23.186811 13062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:23.186942 13062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:23.191123 13062 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:43073
I20260929 19:10:23.191195 13129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:43073 every 8 connection(s)
I20260929 19:10:23.191679 13062 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:10:23.192456 13130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:23.193126 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 13062
I20260929 19:10:23.193223 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:10:23.193403 10160 external_mini_cluster.cc:1428] Setting key 26c22ab3caed375d087e54ee8b8cd548
I20260929 19:10:23.195470 13130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314: Bootstrap starting.
I20260929 19:10:23.196192 13130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:23.196462 13130 log.cc:824] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:23.197268 13130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314: No bootstrap required, opened a new log
I20260929 19:10:23.199828 13130 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ff55c9f0a4344cdbc6610e15d97e314" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43073 } }
I20260929 19:10:23.200035 13130 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:23.200069 13130 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ff55c9f0a4344cdbc6610e15d97e314, State: Initialized, Role: FOLLOWER
I20260929 19:10:23.200199 13130 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ff55c9f0a4344cdbc6610e15d97e314" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43073 } }
I20260929 19:10:23.200275 13130 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:10:23.200304 13130 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:10:23.200353 13130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314 [term 0 FOLLOWER]: Advancing to term 1
Sep 29 19:10:23 dist-test-slave-wftq krb5kdc[13049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:23.200966 13130 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ff55c9f0a4344cdbc6610e15d97e314" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43073 } }
I20260929 19:10:23.201064 13130 leader_election.cc:302] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ff55c9f0a4344cdbc6610e15d97e314; no voters:
I20260929 19:10:23.201259 13130 leader_election.cc:288] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:10:23.201310 13135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:23.201448 13135 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314 [term 1 LEADER]: Becoming Leader. State: Replica: 6ff55c9f0a4344cdbc6610e15d97e314, State: Running, Role: LEADER
I20260929 19:10:23.201572 13130 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:23.201598 13135 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ff55c9f0a4344cdbc6610e15d97e314" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43073 } }
I20260929 19:10:23.201941 13135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ff55c9f0a4344cdbc6610e15d97e314" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ff55c9f0a4344cdbc6610e15d97e314" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43073 } } }
I20260929 19:10:23.202143 13135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:23.202353 13135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ff55c9f0a4344cdbc6610e15d97e314. Latest consensus state: current_term: 1 leader_uuid: "6ff55c9f0a4344cdbc6610e15d97e314" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ff55c9f0a4344cdbc6610e15d97e314" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43073 } } }
I20260929 19:10:23.202464 13135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:23.202623 13144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:23.203507 13144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:10:23.204957 13133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:23.193932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55620 (local address 127.9.236.62:43073)
0929 19:10:23.194170 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:23.194179 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:23.194206 (+ 27us) server_negotiation.cc:407] Connection header received
0929 19:10:23.194762 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:23.194768 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:23.194817 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:23.194941 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:23.195566 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:23.196542 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:23.197700 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:23.198130 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:23.198219 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:23.201259 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:23.201263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:23.201263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:23.201275 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:23.203236 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:23.203645 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:23.203649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:23.203649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:23.203752 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:23.203951 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:23.203952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:23.203953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:23.204250 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:23.204394 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:23.204596 (+ 202us) server_negotiation.cc:299] Negotiation successful
0929 19:10:23.204739 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":122,"threads_started":1}
I20260929 19:10:23.205681 13144 catalog_manager.cc:1347] Generated new cluster ID: f753cd6f8e3041d8a50ba0572866e29c
I20260929 19:10:23.205747 13144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:10:23.205938 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:10:23.222657 13144 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:10:23.223475 13144 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:23.227473 13144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ff55c9f0a4344cdbc6610e15d97e314: Generated new TSK 0
I20260929 19:10:23.227632 13144 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:10:23.227908 13133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:23.223321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55630 (local address 127.9.236.62:43073)
0929 19:10:23.223376 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:23.223378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:23.223578 (+ 200us) server_negotiation.cc:407] Connection header received
0929 19:10:23.223616 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:23.223617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:23.223629 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:23.223701 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:23.224182 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:23.224934 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:23.226105 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:23.226440 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:23.226505 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:23.227360 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:23.227362 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:23.227362 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:10:23.227633 (+ 271us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:23.227659 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:10:23.227667 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:23.227738 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:23.227835 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 19:10:23.227860 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 19:10:23.230243 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 13062
2026-09-29T19:10:23Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1259 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:10:23 dist-test-slave-wftq krb5kdc[13163](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:10:23 dist-test-slave-wftq krb5kdc[13163](info): set up 2 sockets
Sep 29 19:10:23 dist-test-slave-wftq krb5kdc[13163](info): commencing operation
krb5kdc: starting...
W20260929 19:10:24.184975 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:10:24 dist-test-slave-wftq krb5kdc[13163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:10:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:10:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:10:24.303480 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:43825
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:43593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:43825
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:24.384959 13176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:24.385229 13176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:24.385255 13176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:24.386384 13176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:24.386443 13176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:24.386457 13176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:24.386468 13176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:24.386478 13176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:24.387885 13176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:43593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:43825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:43825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:24.388082 13176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:24.388259 13176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:24.393362 13181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:24.393447 13176 server_base.cc:1034] running on GCE node
W20260929 19:10:24.393383 13184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:24.393525 13182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:24.393817 13176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:24.394093 13176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:24.395251 13176 hybrid_clock.cc:648] HybridClock initialized: now 1790709024395245 us; error 35 us; skew 500 ppm
Sep 29 19:10:24 dist-test-slave-wftq krb5kdc[13163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709024, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:24.397738 13176 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:10:24.398190 13176 webserver.cc:460] Webserver started at http://127.9.236.62:34197/ using document root <none> and password file <none>
I20260929 19:10:24.398401 13176 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:24.398450 13176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:24.398569 13176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:24.399560 13176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "787680bce1f74bd6b1228d1f99d525d7"
format_stamp: "Formatted at 2026-09-29 19:10:24 on dist-test-slave-wftq"
server_key: "212de9f44df7307d38b8fafb298e2517"
server_key_iv: "0aa8e11d735dab76271820dd24d4b319"
server_key_version: "encryptionkey@0"
I20260929 19:10:24.399875 13176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "787680bce1f74bd6b1228d1f99d525d7"
format_stamp: "Formatted at 2026-09-29 19:10:24 on dist-test-slave-wftq"
server_key: "212de9f44df7307d38b8fafb298e2517"
server_key_iv: "0aa8e11d735dab76271820dd24d4b319"
server_key_version: "encryptionkey@0"
I20260929 19:10:24.401216 13176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:10:24.402056 13191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:24.402271 13176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:24.402357 13176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "787680bce1f74bd6b1228d1f99d525d7"
format_stamp: "Formatted at 2026-09-29 19:10:24 on dist-test-slave-wftq"
server_key: "212de9f44df7307d38b8fafb298e2517"
server_key_iv: "0aa8e11d735dab76271820dd24d4b319"
server_key_version: "encryptionkey@0"
I20260929 19:10:24.402428 13176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:24.416320 13176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:24.418476 13176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:24.418607 13176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:24.422729 13176 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:43825
I20260929 19:10:24.422793 13243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:43825 every 8 connection(s)
I20260929 19:10:24.423103 13176 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:10:24.424209 13244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:24.426965 13244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7: Bootstrap starting.
I20260929 19:10:24.427595 13244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:24.427906 13244 log.cc:824] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:24.428555 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 13176
I20260929 19:10:24.428697 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:10:24.428903 10160 external_mini_cluster.cc:1428] Setting key 0b07c3de67dd1a571292d0d103a40f3d
I20260929 19:10:24.429042 13244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7: No bootstrap required, opened a new log
I20260929 19:10:24.431358 13244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787680bce1f74bd6b1228d1f99d525d7" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43825 } }
I20260929 19:10:24.431531 13244 raft_consensus.cc:374] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:24.431565 13244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787680bce1f74bd6b1228d1f99d525d7, State: Initialized, Role: FOLLOWER
I20260929 19:10:24.431707 13244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787680bce1f74bd6b1228d1f99d525d7" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43825 } }
I20260929 19:10:24.431792 13244 raft_consensus.cc:388] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:10:24.431834 13244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:10:24.431883 13244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:24.432674 13244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787680bce1f74bd6b1228d1f99d525d7" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43825 } }
I20260929 19:10:24.432778 13244 leader_election.cc:302] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 787680bce1f74bd6b1228d1f99d525d7; no voters:
I20260929 19:10:24.432930 13244 leader_election.cc:288] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:10:24.432974 13249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:24.433159 13249 raft_consensus.cc:686] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7 [term 1 LEADER]: Becoming Leader. State: Replica: 787680bce1f74bd6b1228d1f99d525d7, State: Running, Role: LEADER
I20260929 19:10:24.433233 13244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:24.433310 13249 consensus_queue.cc:237] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787680bce1f74bd6b1228d1f99d525d7" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43825 } }
I20260929 19:10:24.433825 13251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 787680bce1f74bd6b1228d1f99d525d7. Latest consensus state: current_term: 1 leader_uuid: "787680bce1f74bd6b1228d1f99d525d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787680bce1f74bd6b1228d1f99d525d7" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43825 } } }
I20260929 19:10:24.433908 13251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:24.434120 13250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "787680bce1f74bd6b1228d1f99d525d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787680bce1f74bd6b1228d1f99d525d7" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43825 } } }
I20260929 19:10:24.434415 13250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:24.434643 13258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:24.435191 13258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:10:24.436738 13258 catalog_manager.cc:1347] Generated new cluster ID: 8571d0be741a48afbc3483a71c923b2a
I20260929 19:10:24.436792 13258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:10:24 dist-test-slave-wftq krb5kdc[13163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:24.441789 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:24.429557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49186 (local address 127.9.236.62:43825)
0929 19:10:24.429809 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:24.429819 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:24.429865 (+ 46us) server_negotiation.cc:407] Connection header received
0929 19:10:24.430594 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:24.430611 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:24.430676 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:24.430842 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:24.431467 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:24.433017 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:24.434140 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:24.434732 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:24.434837 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:24.437904 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:24.437909 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:24.437909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:24.437928 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:24.439901 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:24.440295 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:24.440298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:24.440299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:24.440394 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:24.440545 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:24.440546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:24.440546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:24.440838 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:24.440981 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:24.441439 (+ 458us) server_negotiation.cc:299] Negotiation successful
0929 19:10:24.441579 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":143,"threads_started":1}
I20260929 19:10:24.457597 13258 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:10:24.458227 13258 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:24.463752 13258 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 787680bce1f74bd6b1228d1f99d525d7: Generated new TSK 0
I20260929 19:10:24.463903 13258 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.236.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
I20260929 19:10:24.500511 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:43825
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:43593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:24.607637 13270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:24.607839 13270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:24.607861 13270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:24.608991 13270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:24.609035 13270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:24.609061 13270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:10:24.610503 13270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:43593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:43825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:24.610719 13270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:24.610946 13270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:24.616226 13276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:24.616211 13275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:24.616477 13270 server_base.cc:1034] running on GCE node
W20260929 19:10:24.616236 13278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:24.616838 13270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:24.617156 13270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:24.618331 13270 hybrid_clock.cc:648] HybridClock initialized: now 1790709024618308 us; error 49 us; skew 500 ppm
Sep 29 19:10:24 dist-test-slave-wftq krb5kdc[13163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709024, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:24.621309 13270 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:10:24.621795 13270 webserver.cc:460] Webserver started at http://127.9.236.1:42925/ using document root <none> and password file <none>
I20260929 19:10:24.622016 13270 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:24.622064 13270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:24.622177 13270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:24.623286 13270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "cb24ed5ef54947d6915272fee3d956a0"
format_stamp: "Formatted at 2026-09-29 19:10:24 on dist-test-slave-wftq"
server_key: "92cfbf0d6a4d4f3518a2be3ab082f799"
server_key_iv: "18c9759d1390f394894181a1162b72eb"
server_key_version: "encryptionkey@0"
I20260929 19:10:24.623602 13270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "cb24ed5ef54947d6915272fee3d956a0"
format_stamp: "Formatted at 2026-09-29 19:10:24 on dist-test-slave-wftq"
server_key: "92cfbf0d6a4d4f3518a2be3ab082f799"
server_key_iv: "18c9759d1390f394894181a1162b72eb"
server_key_version: "encryptionkey@0"
I20260929 19:10:24.624966 13270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:10:24.625861 13285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:24.626080 13270 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:24.626178 13270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "cb24ed5ef54947d6915272fee3d956a0"
format_stamp: "Formatted at 2026-09-29 19:10:24 on dist-test-slave-wftq"
server_key: "92cfbf0d6a4d4f3518a2be3ab082f799"
server_key_iv: "18c9759d1390f394894181a1162b72eb"
server_key_version: "encryptionkey@0"
I20260929 19:10:24.626277 13270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:24.646730 13270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:24.648744 13270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:24.648859 13270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:24.649085 13270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:24.649449 13270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:24.649487 13270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:24.649507 13270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:24.649537 13270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:24.655683 13270 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:39887
I20260929 19:10:24.655787 13398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:39887 every 8 connection(s)
I20260929 19:10:24.656066 13270 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:10:24.657014 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 13270
I20260929 19:10:24.657171 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:10:24.657437 10160 external_mini_cluster.cc:1428] Setting key b8e595274067651f328894109aa8ddb3
Sep 29 19:10:24 dist-test-slave-wftq krb5kdc[13163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709024, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.236.2@KRBTEST.COM; defaulting to no policy
I20260929 19:10:24.667057 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:24.656880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:45543 (local address 127.9.236.62:43825)
0929 19:10:24.656986 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:24.656989 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:24.657296 (+ 307us) server_negotiation.cc:407] Connection header received
0929 19:10:24.658034 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:24.658035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:24.658048 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:24.658112 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:24.659209 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:24.659767 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:24.661280 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:24.661501 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:24.661571 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:24.664233 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:24.664234 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:24.664234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:24.664243 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:24.665594 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:24.666076 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:24.666077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:24.666077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:24.666140 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:24.666423 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:24.666423 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:24.666423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:24.666540 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:24.666598 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:24.666872 (+ 274us) server_negotiation.cc:299] Negotiation successful
0929 19:10:24.666963 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 19:10:24.667290 13401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:24.656801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:43825 (local address 127.9.236.1:45543)
0929 19:10:24.657082 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:24.657161 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:24.657933 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:24.658214 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:24.658218 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:24.658863 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:24.659091 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:24.659096 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:24.659860 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:24.659861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:24.661174 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:24.661177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:24.661594 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:24.661595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:24.661762 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:24.662327 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:24.662339 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:24.664127 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:24.665681 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:24.665688 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:24.665691 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:24.665973 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:24.666223 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:24.666224 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:24.666224 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:24.666353 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:24.666599 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:24.666604 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:24.666748 (+ 144us) client_negotiation.cc:769] Sending connection context
0929 19:10:24.666877 (+ 129us) client_negotiation.cc:240] Negotiation successful
0929 19:10:24.667038 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":93,"threads_started":1}
I20260929 19:10:24.667876 13399 heartbeater.cc:347] Connected to a master server at 127.9.236.62:43825
I20260929 19:10:24.667974 13399 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:24.668174 13399 heartbeater.cc:511] Master 127.9.236.62:43825 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.2@KRBTEST.COM" created.
I20260929 19:10:24.668876 13208 ts_manager.cc:194] Registered new tserver with Master: cb24ed5ef54947d6915272fee3d956a0 (127.9.236.1:39887)
I20260929 19:10:24.669633 13208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:45543
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
I20260929 19:10:24.684885 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:43825
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:43593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:24.772413 13404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:24.772637 13404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:24.772677 13404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:24.773933 13404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:24.773974 13404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:24.774008 13404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:10:24.775446 13404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:43593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:43825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:24.775700 13404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:24.775928 13404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:24.781131 13409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:24.781121 13412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:24.781358 13404 server_base.cc:1034] running on GCE node
W20260929 19:10:24.781384 13410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:24.781646 13404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:24.781888 13404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:24.783044 13404 hybrid_clock.cc:648] HybridClock initialized: now 1790709024783028 us; error 38 us; skew 500 ppm
Sep 29 19:10:24 dist-test-slave-wftq krb5kdc[13163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709024, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:24.785640 13404 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:10:24.786135 13404 webserver.cc:460] Webserver started at http://127.9.236.2:40525/ using document root <none> and password file <none>
I20260929 19:10:24.786355 13404 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:24.786412 13404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:24.786537 13404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:24.787602 13404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "8319da85fbc54feab851bb0a2d38d6b8"
format_stamp: "Formatted at 2026-09-29 19:10:24 on dist-test-slave-wftq"
server_key: "36af6e7622a920dc707052c55e24cc24"
server_key_iv: "68967ba60cad2610cfc0267b1e30a336"
server_key_version: "encryptionkey@0"
I20260929 19:10:24.787948 13404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "8319da85fbc54feab851bb0a2d38d6b8"
format_stamp: "Formatted at 2026-09-29 19:10:24 on dist-test-slave-wftq"
server_key: "36af6e7622a920dc707052c55e24cc24"
server_key_iv: "68967ba60cad2610cfc0267b1e30a336"
server_key_version: "encryptionkey@0"
I20260929 19:10:24.789294 13404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:10:24.790132 13419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:24.790304 13404 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:10:24.790385 13404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "8319da85fbc54feab851bb0a2d38d6b8"
format_stamp: "Formatted at 2026-09-29 19:10:24 on dist-test-slave-wftq"
server_key: "36af6e7622a920dc707052c55e24cc24"
server_key_iv: "68967ba60cad2610cfc0267b1e30a336"
server_key_version: "encryptionkey@0"
I20260929 19:10:24.790453 13404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:24.826143 13404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:24.828183 13404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:24.828342 13404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:24.828619 13404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:24.828991 13404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:24.829035 13404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:24.829063 13404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:24.829093 13404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:24.836184 13404 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:44787
I20260929 19:10:24.836256 13532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:44787 every 8 connection(s)
I20260929 19:10:24.836552 13404 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:10:24.840703 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 13404
I20260929 19:10:24.840821 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:10:24.841022 10160 external_mini_cluster.cc:1428] Setting key 1c85445c08830af65a5a78ef740ee60e
Sep 29 19:10:24 dist-test-slave-wftq krb5kdc[13163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709024, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:24.846127 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:24.837344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:55651 (local address 127.9.236.62:43825)
0929 19:10:24.837422 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:24.837424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:24.837592 (+ 168us) server_negotiation.cc:407] Connection header received
0929 19:10:24.838190 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:24.838191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:24.838202 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:24.838255 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:24.839208 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:24.839695 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:24.840924 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:24.841183 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:24.841253 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:24.843696 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:24.843698 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:24.843698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:24.843705 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:24.844844 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:24.845276 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:24.845277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:24.845278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:24.845330 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:24.845538 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:24.845538 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:24.845539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:24.845650 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:24.845702 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:24.845961 (+ 259us) server_negotiation.cc:299] Negotiation successful
0929 19:10:24.846036 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 19:10:24.846364 13535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:24.837294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:43825 (local address 127.9.236.2:55651)
0929 19:10:24.837457 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:24.837496 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:24.838102 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:24.838325 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:24.838329 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:24.838921 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:24.839134 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:24.839139 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:24.839781 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:24.839782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:24.840779 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:24.840782 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:24.841259 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:24.841260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:24.841373 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:24.841952 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:24.841961 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:24.843605 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:24.844917 (+ 1312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:24.844923 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:24.844926 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:24.845220 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:24.845380 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:24.845380 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:24.845381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:24.845487 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:24.845701 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:24.845707 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:24.845849 (+ 142us) client_negotiation.cc:769] Sending connection context
0929 19:10:24.845937 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 19:10:24.846128 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":73,"threads_started":1}
I20260929 19:10:24.846832 13533 heartbeater.cc:347] Connected to a master server at 127.9.236.62:43825
I20260929 19:10:24.846954 13533 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:24.847172 13533 heartbeater.cc:511] Master 127.9.236.62:43825 requested a full tablet report, sending...
I20260929 19:10:24.847560 13208 ts_manager.cc:194] Registered new tserver with Master: 8319da85fbc54feab851bb0a2d38d6b8 (127.9.236.2:44787)
I20260929 19:10:24.847922 13208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:55651
WARNING: no policy specified for kudu/127.9.236.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
I20260929 19:10:24.866113 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:43825
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:43593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:24.953481 13538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:24.953704 13538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:24.953727 13538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:24.954905 13538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:24.954946 13538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:24.954972 13538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:10:24.956372 13538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:43593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:43825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:24.956563 13538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:24.956758 13538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:24.961859 13546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:24.961872 13543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:24.961859 13544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:24.962239 13538 server_base.cc:1034] running on GCE node
I20260929 19:10:24.962440 13538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:24.962694 13538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:24.963842 13538 hybrid_clock.cc:648] HybridClock initialized: now 1790709024963821 us; error 35 us; skew 500 ppm
Sep 29 19:10:24 dist-test-slave-wftq krb5kdc[13163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709024, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:24.971650 13538 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:10:24.972124 13538 webserver.cc:460] Webserver started at http://127.9.236.3:43931/ using document root <none> and password file <none>
I20260929 19:10:24.972335 13538 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:24.972386 13538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:24.972512 13538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:24.973613 13538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "3fc1f51e2bd74d66973003d7ce70f092"
format_stamp: "Formatted at 2026-09-29 19:10:24 on dist-test-slave-wftq"
server_key: "edf87f98d747ba54774a47de305bd294"
server_key_iv: "aab4cdcd7c273b5b55757a7b84e06485"
server_key_version: "encryptionkey@0"
I20260929 19:10:24.973934 13538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "3fc1f51e2bd74d66973003d7ce70f092"
format_stamp: "Formatted at 2026-09-29 19:10:24 on dist-test-slave-wftq"
server_key: "edf87f98d747ba54774a47de305bd294"
server_key_iv: "aab4cdcd7c273b5b55757a7b84e06485"
server_key_version: "encryptionkey@0"
I20260929 19:10:24.975198 13538 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:10:24.975907 13553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:24.976089 13538 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:10:24.976174 13538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "3fc1f51e2bd74d66973003d7ce70f092"
format_stamp: "Formatted at 2026-09-29 19:10:24 on dist-test-slave-wftq"
server_key: "edf87f98d747ba54774a47de305bd294"
server_key_iv: "aab4cdcd7c273b5b55757a7b84e06485"
server_key_version: "encryptionkey@0"
I20260929 19:10:24.976234 13538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:24.989413 13538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:24.991760 13538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:24.991914 13538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:24.992141 13538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:24.992475 13538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:24.992511 13538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:24.992537 13538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:24.992561 13538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:24.999471 13538 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:44319
I20260929 19:10:24.999521 13666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:44319 every 8 connection(s)
I20260929 19:10:24.999887 13538 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:10:25.001294 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 13538
I20260929 19:10:25.001379 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:10:25.001547 10160 external_mini_cluster.cc:1428] Setting key c7d255b2fd6d907e5d606df41a71f8be
Sep 29 19:10:25 dist-test-slave-wftq krb5kdc[13163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709024, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:25.008996 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:25.000718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:47887 (local address 127.9.236.62:43825)
0929 19:10:25.000787 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:25.000789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:25.000986 (+ 197us) server_negotiation.cc:407] Connection header received
0929 19:10:25.001576 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:25.001578 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:25.001590 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:25.001641 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:25.002752 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:25.003244 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:25.004199 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:25.004427 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:25.004500 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:25.006686 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:25.006688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:25.006688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:25.006695 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:25.007820 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:25.008186 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:25.008186 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:25.008187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:25.008236 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:25.008434 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:25.008434 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:25.008435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:25.008540 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:25.008593 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:25.008834 (+ 241us) server_negotiation.cc:299] Negotiation successful
0929 19:10:25.008907 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 19:10:25.009197 13669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:25.000667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:43825 (local address 127.9.236.3:47887)
0929 19:10:25.000859 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:25.000898 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:25.001470 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:25.001714 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:25.001718 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:25.002455 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:25.002673 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:25.002678 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:25.003321 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:25.003322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:25.004122 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:25.004125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:25.004502 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:25.004503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:25.004635 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:25.005152 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:25.005162 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:25.006605 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:25.007889 (+ 1284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:25.007895 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:25.007897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:25.008106 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:25.008288 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:25.008289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:25.008289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:25.008381 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:25.008599 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:25.008604 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:25.008724 (+ 120us) client_negotiation.cc:769] Sending connection context
0929 19:10:25.008809 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 19:10:25.008956 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":95,"threads_started":1}
I20260929 19:10:25.009727 13667 heartbeater.cc:347] Connected to a master server at 127.9.236.62:43825
I20260929 19:10:25.009814 13667 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:25.009994 13667 heartbeater.cc:511] Master 127.9.236.62:43825 requested a full tablet report, sending...
I20260929 19:10:25.010394 13208 ts_manager.cc:194] Registered new tserver with Master: 3fc1f51e2bd74d66973003d7ce70f092 (127.9.236.3:44319)
I20260929 19:10:25.010819 13208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:47887
I20260929 19:10:25.013281 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 19:10:25 dist-test-slave-wftq krb5kdc[13163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709025, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 29 19:10:25 dist-test-slave-wftq krb5kdc[13163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709025, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:25.036227 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:25.029397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47962 (local address 127.9.236.62:43825)
0929 19:10:25.029477 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:25.029479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:25.029608 (+ 129us) server_negotiation.cc:407] Connection header received
0929 19:10:25.029674 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:25.029675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:25.029685 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:25.029731 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:25.030328 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:25.030811 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:25.031552 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:25.031742 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:25.031789 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:25.034114 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:25.034116 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:25.034116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:25.034122 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:25.035342 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:25.035636 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:25.035636 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:25.035637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:25.035679 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:25.035824 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:25.035824 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:25.035824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:25.035951 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:25.036023 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:25.036105 (+ 82us) server_negotiation.cc:299] Negotiation successful
0929 19:10:25.036161 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260929 19:10:25.036520 13208 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:47962
I20260929 19:10:25.037137 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 13270
I20260929 19:10:25.041990 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 13404
I20260929 19:10:25.046211 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 13538
I20260929 19:10:25.050693 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 13176
2026-09-29T19:10:25Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1820 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:10:25 dist-test-slave-wftq krb5kdc[13679](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:10:25 dist-test-slave-wftq krb5kdc[13679](info): set up 2 sockets
Sep 29 19:10:25 dist-test-slave-wftq krb5kdc[13679](info): commencing operation
krb5kdc: starting...
W20260929 19:10:26.004963 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:10:26 dist-test-slave-wftq krb5kdc[13679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:10:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:10:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:10:26.125694 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:41953
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:41953
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:26.208099 13692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:26.208312 13692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:26.208341 13692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:26.209506 13692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:26.209548 13692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:26.209568 13692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:26.209583 13692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:26.209597 13692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:26.211069 13692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:41953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:41953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:26.211305 13692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:26.211509 13692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:26.216651 13698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:26.216702 13697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:26.216725 13692 server_base.cc:1034] running on GCE node
W20260929 19:10:26.216668 13700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:26.217263 13692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:26.217582 13692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:26.218734 13692 hybrid_clock.cc:648] HybridClock initialized: now 1790709026218719 us; error 42 us; skew 500 ppm
Sep 29 19:10:26 dist-test-slave-wftq krb5kdc[13679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709026, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:26.221306 13692 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:10:26.221753 13692 webserver.cc:460] Webserver started at http://127.9.236.62:41137/ using document root <none> and password file <none>
I20260929 19:10:26.221957 13692 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:26.222007 13692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:26.222123 13692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:26.223125 13692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "914623cb71c5445da5fc96a7af94752d"
format_stamp: "Formatted at 2026-09-29 19:10:26 on dist-test-slave-wftq"
server_key: "10cef99e5caad2ed9de0cc6d6099b8a1"
server_key_iv: "6b8c86fbd53c476657f3da02b8526d00"
server_key_version: "encryptionkey@0"
I20260929 19:10:26.223443 13692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "914623cb71c5445da5fc96a7af94752d"
format_stamp: "Formatted at 2026-09-29 19:10:26 on dist-test-slave-wftq"
server_key: "10cef99e5caad2ed9de0cc6d6099b8a1"
server_key_iv: "6b8c86fbd53c476657f3da02b8526d00"
server_key_version: "encryptionkey@0"
I20260929 19:10:26.224687 13692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:10:26.225482 13707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:26.225677 13692 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:26.225761 13692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "914623cb71c5445da5fc96a7af94752d"
format_stamp: "Formatted at 2026-09-29 19:10:26 on dist-test-slave-wftq"
server_key: "10cef99e5caad2ed9de0cc6d6099b8a1"
server_key_iv: "6b8c86fbd53c476657f3da02b8526d00"
server_key_version: "encryptionkey@0"
I20260929 19:10:26.225821 13692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:26.234232 13692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:26.236078 13692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:26.236203 13692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:26.240399 13692 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:41953
I20260929 19:10:26.240453 13759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:41953 every 8 connection(s)
I20260929 19:10:26.240899 13692 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:10:26.241686 13760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:26.244201 13760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d: Bootstrap starting.
I20260929 19:10:26.244819 13760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:26.245091 13760 log.cc:824] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:26.245958 13760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d: No bootstrap required, opened a new log
I20260929 19:10:26.247640 13760 raft_consensus.cc:348] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "914623cb71c5445da5fc96a7af94752d" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 41953 } }
I20260929 19:10:26.247802 13760 raft_consensus.cc:374] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:26.247822 13760 raft_consensus.cc:729] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 914623cb71c5445da5fc96a7af94752d, State: Initialized, Role: FOLLOWER
I20260929 19:10:26.247956 13760 consensus_queue.cc:260] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "914623cb71c5445da5fc96a7af94752d" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 41953 } }
I20260929 19:10:26.248031 13760 raft_consensus.cc:388] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:10:26.248063 13760 raft_consensus.cc:482] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:10:26.248098 13760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:26.248998 13760 raft_consensus.cc:504] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "914623cb71c5445da5fc96a7af94752d" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 41953 } }
I20260929 19:10:26.249089 13760 leader_election.cc:302] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 914623cb71c5445da5fc96a7af94752d; no voters:
I20260929 19:10:26.249292 13760 leader_election.cc:288] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:10:26.249372 13763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:26.249547 13760 sys_catalog.cc:564] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:26.249606 13763 raft_consensus.cc:686] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d [term 1 LEADER]: Becoming Leader. State: Replica: 914623cb71c5445da5fc96a7af94752d, State: Running, Role: LEADER
I20260929 19:10:26.249745 13763 consensus_queue.cc:237] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "914623cb71c5445da5fc96a7af94752d" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 41953 } }
I20260929 19:10:26.250205 13764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "914623cb71c5445da5fc96a7af94752d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "914623cb71c5445da5fc96a7af94752d" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 41953 } } }
I20260929 19:10:26.250217 13765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 914623cb71c5445da5fc96a7af94752d. Latest consensus state: current_term: 1 leader_uuid: "914623cb71c5445da5fc96a7af94752d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "914623cb71c5445da5fc96a7af94752d" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 41953 } } }
I20260929 19:10:26.250325 13764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:26.250329 13765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:26.250877 13772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:26.250946 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 13692
I20260929 19:10:26.251046 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:10:26.251231 10160 external_mini_cluster.cc:1428] Setting key 3ae4d3b47680f8c7b7cae6474ab3928b
I20260929 19:10:26.251616 13772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:10:26.253656 13772 catalog_manager.cc:1347] Generated new cluster ID: 4add1518ebca4c07ae6edc5ce131a58b
I20260929 19:10:26.253705 13772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:10:26 dist-test-slave-wftq krb5kdc[13679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:26.269047 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:26.251821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59234 (local address 127.9.236.62:41953)
0929 19:10:26.252115 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:26.252125 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:26.252707 (+ 582us) server_negotiation.cc:407] Connection header received
0929 19:10:26.252859 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:26.252867 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:26.252931 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:26.253166 (+ 235us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:26.257944 (+ 4778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:26.259723 (+ 1779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:26.260917 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:26.261657 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:26.261762 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:26.264991 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:26.264999 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:26.264999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:26.265016 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:26.267145 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:26.267599 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:26.267602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:26.267603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:26.267763 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:26.267971 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:26.267972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:26.267972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:26.268340 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:26.268522 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:26.268602 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 19:10:26.268778 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":106,"threads_started":1}
I20260929 19:10:26.284931 13772 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:10:26.285487 13772 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:26.287771 13772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 914623cb71c5445da5fc96a7af94752d: Generated new TSK 0
I20260929 19:10:26.287919 13772 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.236.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
I20260929 19:10:26.318181 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:41953
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:26.400213 13786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:26.400426 13786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:26.400456 13786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:26.401666 13786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:26.401707 13786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:26.401741 13786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:10:26.403251 13786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:41953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:26.403460 13786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:26.403638 13786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:26.408610 13792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:26.408699 13786 server_base.cc:1034] running on GCE node
W20260929 19:10:26.408622 13794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:26.408632 13791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:26.409070 13786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:26.409305 13786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:26.410475 13786 hybrid_clock.cc:648] HybridClock initialized: now 1790709026410450 us; error 37 us; skew 500 ppm
Sep 29 19:10:26 dist-test-slave-wftq krb5kdc[13679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709026, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:26.413036 13786 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:10:26.413506 13786 webserver.cc:460] Webserver started at http://127.9.236.1:39953/ using document root <none> and password file <none>
I20260929 19:10:26.413704 13786 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:26.413753 13786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:26.413864 13786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:26.414836 13786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "e3cde456f7c543069383141bfff61de1"
format_stamp: "Formatted at 2026-09-29 19:10:26 on dist-test-slave-wftq"
server_key: "debe692ad2f5c3723e17000992a7bf91"
server_key_iv: "653103be38c0375d66548fc8402f9cb9"
server_key_version: "encryptionkey@0"
I20260929 19:10:26.415149 13786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "e3cde456f7c543069383141bfff61de1"
format_stamp: "Formatted at 2026-09-29 19:10:26 on dist-test-slave-wftq"
server_key: "debe692ad2f5c3723e17000992a7bf91"
server_key_iv: "653103be38c0375d66548fc8402f9cb9"
server_key_version: "encryptionkey@0"
I20260929 19:10:26.416533 13786 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:10:26.417347 13801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:26.417552 13786 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:26.417639 13786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "e3cde456f7c543069383141bfff61de1"
format_stamp: "Formatted at 2026-09-29 19:10:26 on dist-test-slave-wftq"
server_key: "debe692ad2f5c3723e17000992a7bf91"
server_key_iv: "653103be38c0375d66548fc8402f9cb9"
server_key_version: "encryptionkey@0"
I20260929 19:10:26.417696 13786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:26.427203 13786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:26.429690 13786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:26.429854 13786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:26.430110 13786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:26.430454 13786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:26.430491 13786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:26.430519 13786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:26.430549 13786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:26.437963 13786 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:45213
I20260929 19:10:26.438041 13914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:45213 every 8 connection(s)
I20260929 19:10:26.438365 13786 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:10:26.443964 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 13786
I20260929 19:10:26.444077 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:10:26.444275 10160 external_mini_cluster.cc:1428] Setting key f4944300f8dfe958143d2a23b88d95bb
Sep 29 19:10:26 dist-test-slave-wftq krb5kdc[13679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709026, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:26.448760 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:26.439132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:59379 (local address 127.9.236.62:41953)
0929 19:10:26.439193 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:26.439196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:26.439397 (+ 201us) server_negotiation.cc:407] Connection header received
0929 19:10:26.440326 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:26.440328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:26.440344 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:26.440425 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:26.441064 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:26.441685 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:26.442703 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:26.442910 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:26.442964 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:26.445739 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:26.445740 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:26.445741 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:26.445748 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:26.447210 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:26.447769 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:26.447770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:26.447771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:26.447842 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:26.448087 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:26.448088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:26.448088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:26.448230 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:26.448309 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:26.448580 (+ 271us) server_negotiation.cc:299] Negotiation successful
0929 19:10:26.448670 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 19:10:26.448928 13917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:26.439072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:41953 (local address 127.9.236.1:59379)
0929 19:10:26.439268 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:26.439309 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:26.440215 (+ 906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:26.440494 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:26.440498 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:26.440754 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:26.440962 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:26.440966 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:26.441779 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:26.441780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:26.442614 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:26.442617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:26.442979 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:26.442980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:26.443079 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:26.443636 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:26.443645 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:26.445611 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:26.447369 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:26.447379 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:26.447383 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:26.447687 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:26.447925 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:26.447925 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:26.447926 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:26.448029 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:26.448301 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:26.448304 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:26.448449 (+ 145us) client_negotiation.cc:769] Sending connection context
0929 19:10:26.448537 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 19:10:26.448682 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":118,"threads_started":1}
I20260929 19:10:26.449617 13915 heartbeater.cc:347] Connected to a master server at 127.9.236.62:41953
I20260929 19:10:26.449715 13915 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:26.449914 13915 heartbeater.cc:511] Master 127.9.236.62:41953 requested a full tablet report, sending...
I20260929 19:10:26.450585 13724 ts_manager.cc:194] Registered new tserver with Master: e3cde456f7c543069383141bfff61de1 (127.9.236.1:45213)
I20260929 19:10:26.451525 13724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:59379
WARNING: no policy specified for kudu/127.9.236.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
I20260929 19:10:26.469856 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:41953
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:26.547976 13920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:26.548187 13920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:26.548209 13920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:26.549347 13920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:26.549407 13920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:26.549437 13920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:10:26.550908 13920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:41953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:26.551095 13920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:26.551280 13920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:26.556270 13926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:26.556293 13925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:26.556331 13920 server_base.cc:1034] running on GCE node
W20260929 19:10:26.556255 13928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:26.556706 13920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:26.556917 13920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:26.558054 13920 hybrid_clock.cc:648] HybridClock initialized: now 1790709026558033 us; error 38 us; skew 500 ppm
Sep 29 19:10:26 dist-test-slave-wftq krb5kdc[13679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709026, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:26.560586 13920 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:10:26.561080 13920 webserver.cc:460] Webserver started at http://127.9.236.2:35301/ using document root <none> and password file <none>
I20260929 19:10:26.561336 13920 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:26.561386 13920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:26.561506 13920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:26.562672 13920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "eb56655ae4d24a97a7af64d12ebfb756"
format_stamp: "Formatted at 2026-09-29 19:10:26 on dist-test-slave-wftq"
server_key: "a7cecd079c3ad644fe6021f2ba805d17"
server_key_iv: "768a846a65d811b9a7b36124cb7a139c"
server_key_version: "encryptionkey@0"
I20260929 19:10:26.563063 13920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "eb56655ae4d24a97a7af64d12ebfb756"
format_stamp: "Formatted at 2026-09-29 19:10:26 on dist-test-slave-wftq"
server_key: "a7cecd079c3ad644fe6021f2ba805d17"
server_key_iv: "768a846a65d811b9a7b36124cb7a139c"
server_key_version: "encryptionkey@0"
I20260929 19:10:26.564409 13920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:10:26.565243 13935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:26.565436 13920 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:26.565531 13920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "eb56655ae4d24a97a7af64d12ebfb756"
format_stamp: "Formatted at 2026-09-29 19:10:26 on dist-test-slave-wftq"
server_key: "a7cecd079c3ad644fe6021f2ba805d17"
server_key_iv: "768a846a65d811b9a7b36124cb7a139c"
server_key_version: "encryptionkey@0"
I20260929 19:10:26.565617 13920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:26.580464 13920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:26.582832 13920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:26.582978 13920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:26.583245 13920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:26.583583 13920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:26.583630 13920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:26.583658 13920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:26.583688 13920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:26.591182 13920 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:38161
I20260929 19:10:26.591243 14048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:38161 every 8 connection(s)
I20260929 19:10:26.591593 13920 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:10:26.594684 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 13920
I20260929 19:10:26.594805 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:10:26.595041 10160 external_mini_cluster.cc:1428] Setting key 8de4e72db610fc6ed44a0bd890aa773d
Sep 29 19:10:26 dist-test-slave-wftq krb5kdc[13679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709026, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:26.602562 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:26.592379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:43771 (local address 127.9.236.62:41953)
0929 19:10:26.592474 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:26.592477 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:26.592727 (+ 250us) server_negotiation.cc:407] Connection header received
0929 19:10:26.593752 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:26.593753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:26.593764 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:26.593832 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:26.594561 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:26.595227 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:26.596689 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:26.596894 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:26.596965 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:26.599384 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:26.599386 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:26.599387 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:26.599396 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:26.601154 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:26.601584 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:26.601586 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:26.601586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:26.601674 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:26.601911 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:26.601912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:26.601913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:26.602054 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:26.602130 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:26.602379 (+ 249us) server_negotiation.cc:299] Negotiation successful
0929 19:10:26.602464 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 19:10:26.602769 14051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:26.592317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:41953 (local address 127.9.236.2:43771)
0929 19:10:26.592541 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:26.592584 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:26.593636 (+ 1052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:26.593898 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:26.593902 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:26.594193 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:26.594434 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:26.594439 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:26.595343 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:26.595344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:26.596563 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:26.596567 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:26.596990 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:26.596991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:26.597165 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:26.597771 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:26.597784 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:26.599275 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:26.601263 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:26.601269 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:26.601272 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:26.601500 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:26.601747 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:26.601748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:26.601748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:26.601844 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:26.602121 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:26.602124 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:26.602253 (+ 129us) client_negotiation.cc:769] Sending connection context
0929 19:10:26.602339 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 19:10:26.602470 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":147,"threads_started":1}
I20260929 19:10:26.603302 14049 heartbeater.cc:347] Connected to a master server at 127.9.236.62:41953
I20260929 19:10:26.603399 14049 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:26.603576 14049 heartbeater.cc:511] Master 127.9.236.62:41953 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.236.3@KRBTEST.COM; defaulting to no policy
I20260929 19:10:26.603963 13724 ts_manager.cc:194] Registered new tserver with Master: eb56655ae4d24a97a7af64d12ebfb756 (127.9.236.2:38161)
I20260929 19:10:26.604326 13724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:43771
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
I20260929 19:10:26.620504 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:41953
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:26.700172 14054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:26.700387 14054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:26.700415 14054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:26.701615 14054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:26.701661 14054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:26.701686 14054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:10:26.703167 14054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:41953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:26.703374 14054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:26.703599 14054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:10:26.709256 14054 server_base.cc:1034] running on GCE node
W20260929 19:10:26.709194 14059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:26.709196 14062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:26.709211 14060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:26.709857 14054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:26.710175 14054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:26.711350 14054 hybrid_clock.cc:648] HybridClock initialized: now 1790709026711307 us; error 61 us; skew 500 ppm
Sep 29 19:10:26 dist-test-slave-wftq krb5kdc[13679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709026, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:26.714377 14054 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:10:26.714962 14054 webserver.cc:460] Webserver started at http://127.9.236.3:38029/ using document root <none> and password file <none>
I20260929 19:10:26.715224 14054 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:26.715281 14054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:26.715427 14054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:26.716643 14054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "7b0c0d90fc5a49059bc2ce81fe77bbe1"
format_stamp: "Formatted at 2026-09-29 19:10:26 on dist-test-slave-wftq"
server_key: "13124022d69218910ea1cf2c758f2f11"
server_key_iv: "4c5ef229576aa35aef4a75888cad6822"
server_key_version: "encryptionkey@0"
I20260929 19:10:26.717034 14054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "7b0c0d90fc5a49059bc2ce81fe77bbe1"
format_stamp: "Formatted at 2026-09-29 19:10:26 on dist-test-slave-wftq"
server_key: "13124022d69218910ea1cf2c758f2f11"
server_key_iv: "4c5ef229576aa35aef4a75888cad6822"
server_key_version: "encryptionkey@0"
I20260929 19:10:26.718604 14054 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:26.719552 14069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:26.719779 14054 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:26.719863 14054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "7b0c0d90fc5a49059bc2ce81fe77bbe1"
format_stamp: "Formatted at 2026-09-29 19:10:26 on dist-test-slave-wftq"
server_key: "13124022d69218910ea1cf2c758f2f11"
server_key_iv: "4c5ef229576aa35aef4a75888cad6822"
server_key_version: "encryptionkey@0"
I20260929 19:10:26.719935 14054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:26.743080 14054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:26.745360 14054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:26.745507 14054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:26.745770 14054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:26.746097 14054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:26.746138 14054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:26.746170 14054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:26.746192 14054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:26.753356 14054 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:43951
I20260929 19:10:26.753415 14182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:43951 every 8 connection(s)
I20260929 19:10:26.753750 14054 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:10:26.755726 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 14054
I20260929 19:10:26.755829 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:10:26.756026 10160 external_mini_cluster.cc:1428] Setting key 39386a08fcb832bb248be5065fa5053b
Sep 29 19:10:26 dist-test-slave-wftq krb5kdc[13679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709026, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:26.763947 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:26.754485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:41957 (local address 127.9.236.62:41953)
0929 19:10:26.754565 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:26.754567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:26.754809 (+ 242us) server_negotiation.cc:407] Connection header received
0929 19:10:26.755836 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:26.755838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:26.755852 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:26.755938 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:26.756814 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:26.757479 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:26.758552 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:26.758742 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:26.758802 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:26.761089 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:26.761091 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:26.761091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:26.761133 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:26.762409 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:26.762878 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:26.762879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:26.762880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:26.762940 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:26.763315 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:26.763315 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:26.763315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:26.763421 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:26.763477 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:26.763806 (+ 329us) server_negotiation.cc:299] Negotiation successful
0929 19:10:26.763866 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 19:10:26.764204 14185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:26.754474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:41953 (local address 127.9.236.3:41957)
0929 19:10:26.754669 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:26.754711 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:26.755719 (+ 1008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:26.756023 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:26.756026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:26.756402 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:26.756713 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:26.756721 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:26.757564 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:26.757565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:26.758471 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:26.758474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:26.758854 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:26.758855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:26.758980 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:26.759522 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:26.759531 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:26.760976 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:26.762495 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:26.762500 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:26.762503 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:26.762738 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:26.763055 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:26.763056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:26.763057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:26.763236 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:26.763507 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:26.763513 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:26.763685 (+ 172us) client_negotiation.cc:769] Sending connection context
0929 19:10:26.763788 (+ 103us) client_negotiation.cc:240] Negotiation successful
0929 19:10:26.763938 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":112,"threads_started":1}
I20260929 19:10:26.764746 14183 heartbeater.cc:347] Connected to a master server at 127.9.236.62:41953
I20260929 19:10:26.764848 14183 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:26.765038 14183 heartbeater.cc:511] Master 127.9.236.62:41953 requested a full tablet report, sending...
I20260929 19:10:26.765455 13724 ts_manager.cc:194] Registered new tserver with Master: 7b0c0d90fc5a49059bc2ce81fe77bbe1 (127.9.236.3:43951)
I20260929 19:10:26.765861 13724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:41957
I20260929 19:10:26.767787 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 19:10:26 dist-test-slave-wftq krb5kdc[13679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 19:10:26 dist-test-slave-wftq krb5kdc[13679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.1@KRBTEST.COM
I20260929 19:10:26.791334 14190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:26.783600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50536 (local address 127.9.236.1:45213)
0929 19:10:26.783813 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:26.783819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:26.783831 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:10:26.783860 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:26.783863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:26.783880 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:26.783955 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:26.784481 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:26.785336 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:26.786116 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:26.786462 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:26.786521 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:26.788714 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:26.788722 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:26.788722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:26.788730 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:26.790328 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:26.790643 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:26.790647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:26.790647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:26.790706 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:26.790862 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:26.790863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:26.790863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:26.790989 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:26.791105 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:26.791173 (+ 68us) server_negotiation.cc:299] Negotiation successful
0929 19:10:26.791243 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":103,"threads_started":1}
Sep 29 19:10:26 dist-test-slave-wftq krb5kdc[13679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:26.799712 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:26.792816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59248 (local address 127.9.236.62:41953)
0929 19:10:26.792871 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:26.792872 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:26.792883 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:10:26.792936 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:26.792937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:26.792945 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:26.792994 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:26.793617 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:26.794107 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:26.794852 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:26.795089 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:26.795159 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:26.797376 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:26.797378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:26.797378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:26.797385 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:26.798670 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:26.798987 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:26.798988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:26.798989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:26.799037 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:26.799192 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:26.799193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:26.799193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:26.799329 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:26.799417 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:26.799523 (+ 106us) server_negotiation.cc:299] Negotiation successful
0929 19:10:26.799591 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260929 19:10:26.800061 13724 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59248
I20260929 19:10:26.800483 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 13786
I20260929 19:10:26.805783 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 13920
I20260929 19:10:26.810477 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 14054
I20260929 19:10:26.815032 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 13692
2026-09-29T19:10:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1764 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:10:26 dist-test-slave-wftq krb5kdc[14195](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:10:26 dist-test-slave-wftq krb5kdc[14195](info): set up 2 sockets
Sep 29 19:10:26 dist-test-slave-wftq krb5kdc[14195](info): commencing operation
krb5kdc: starting...
W20260929 19:10:27.769043 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:10:27 dist-test-slave-wftq krb5kdc[14195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:10:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:10:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:10:27.887967 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:34871
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:34871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:27.972743 14208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:27.972975 14208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:27.973006 14208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:27.974262 14208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:27.974309 14208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:27.974326 14208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:27.974342 14208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:27.974356 14208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:27.975955 14208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:34871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:34871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:27.976217 14208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:27.976430 14208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:27.981348 14214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:27.981359 14216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:27.981351 14213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:27.981554 14208 server_base.cc:1034] running on GCE node
I20260929 19:10:27.981812 14208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:27.982082 14208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:27.983235 14208 hybrid_clock.cc:648] HybridClock initialized: now 1790709027983215 us; error 42 us; skew 500 ppm
Sep 29 19:10:27 dist-test-slave-wftq krb5kdc[14195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709027, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:27.985854 14208 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:10:27.986214 14208 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:27.986274 14208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:27.986413 14208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:27.987514 14208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "6f352c2be10a4caea1ff403a6abe8aad"
format_stamp: "Formatted at 2026-09-29 19:10:27 on dist-test-slave-wftq"
server_key: "78133c31414a082eec47be12f2ec27ea"
server_key_iv: "e701bb391fab27ec9e772f76ba3530ac"
server_key_version: "encryptionkey@0"
I20260929 19:10:27.987880 14208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "6f352c2be10a4caea1ff403a6abe8aad"
format_stamp: "Formatted at 2026-09-29 19:10:27 on dist-test-slave-wftq"
server_key: "78133c31414a082eec47be12f2ec27ea"
server_key_iv: "e701bb391fab27ec9e772f76ba3530ac"
server_key_version: "encryptionkey@0"
I20260929 19:10:27.989235 14208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:10:27.990023 14222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:27.990200 14208 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:27.990283 14208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "6f352c2be10a4caea1ff403a6abe8aad"
format_stamp: "Formatted at 2026-09-29 19:10:27 on dist-test-slave-wftq"
server_key: "78133c31414a082eec47be12f2ec27ea"
server_key_iv: "e701bb391fab27ec9e772f76ba3530ac"
server_key_version: "encryptionkey@0"
I20260929 19:10:27.990353 14208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:28.005311 14208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:28.007210 14208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:28.007344 14208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:28.011885 14208 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:34871
I20260929 19:10:28.011936 14274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:34871 every 8 connection(s)
I20260929 19:10:28.012217 14208 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:10:28.012725 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 14208
I20260929 19:10:28.012820 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:10:28.012992 10160 external_mini_cluster.cc:1428] Setting key 5239161b6b602204c66d9438d8c60dc0
I20260929 19:10:28.013294 14275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:28.016281 14275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad: Bootstrap starting.
I20260929 19:10:28.016881 14275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:28.017175 14275 log.cc:824] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:28.017797 14275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad: No bootstrap required, opened a new log
Sep 29 19:10:28 dist-test-slave-wftq krb5kdc[14195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:28.019797 14275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f352c2be10a4caea1ff403a6abe8aad" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 34871 } }
I20260929 19:10:28.019972 14275 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:28.020018 14275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f352c2be10a4caea1ff403a6abe8aad, State: Initialized, Role: FOLLOWER
I20260929 19:10:28.020156 14275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f352c2be10a4caea1ff403a6abe8aad" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 34871 } }
I20260929 19:10:28.020239 14275 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:10:28.020279 14275 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:10:28.020327 14275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:28.020956 14275 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f352c2be10a4caea1ff403a6abe8aad" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 34871 } }
I20260929 19:10:28.021064 14275 leader_election.cc:302] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f352c2be10a4caea1ff403a6abe8aad; no voters:
I20260929 19:10:28.021274 14275 leader_election.cc:288] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:10:28.021350 14280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:28.021550 14280 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad [term 1 LEADER]: Becoming Leader. State: Replica: 6f352c2be10a4caea1ff403a6abe8aad, State: Running, Role: LEADER
I20260929 19:10:28.021552 14275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:28.021744 14280 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f352c2be10a4caea1ff403a6abe8aad" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 34871 } }
I20260929 19:10:28.022234 14281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f352c2be10a4caea1ff403a6abe8aad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f352c2be10a4caea1ff403a6abe8aad" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 34871 } } }
I20260929 19:10:28.022403 14281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:28.022713 14289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:28.023239 14282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f352c2be10a4caea1ff403a6abe8aad. Latest consensus state: current_term: 1 leader_uuid: "6f352c2be10a4caea1ff403a6abe8aad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f352c2be10a4caea1ff403a6abe8aad" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 34871 } } }
I20260929 19:10:28.023350 14282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:28.023489 14289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:10:28.024648 14278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:28.013520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43192 (local address 127.9.236.62:34871)
0929 19:10:28.013733 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:28.013745 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:28.013772 (+ 27us) server_negotiation.cc:407] Connection header received
0929 19:10:28.014543 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:28.014550 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:28.014639 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:28.014779 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:28.015405 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.016351 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.017146 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.017618 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.017721 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:28.020439 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:28.020444 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:28.020444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:28.020459 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:28.022427 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:28.022848 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:28.022851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:28.022852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:28.023632 (+ 780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:28.023820 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:28.023821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:28.023821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:28.024065 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:28.024224 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:28.024290 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 19:10:28.024425 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":107,"threads_started":1}
I20260929 19:10:28.025270 14289 catalog_manager.cc:1347] Generated new cluster ID: 7ad243ede66f47789d6dcc2975c13352
I20260929 19:10:28.025317 14289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:10:28.042960 14289 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:10:28.043740 14289 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:28.048933 14289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad: Generated new TSK 0
I20260929 19:10:28.049155 14289 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.236.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
I20260929 19:10:28.075496 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:34871
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:28.156836 14301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:28.157040 14301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:28.157063 14301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:28.158227 14301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:28.158277 14301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:28.158301 14301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:10:28.159771 14301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:34871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:28.159962 14301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:28.160131 14301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:28.164884 14309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:28.164881 14307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:28.164891 14306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:28.165242 14301 server_base.cc:1034] running on GCE node
I20260929 19:10:28.165520 14301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:28.165822 14301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:28.167016 14301 hybrid_clock.cc:648] HybridClock initialized: now 1790709028166996 us; error 38 us; skew 500 ppm
Sep 29 19:10:28 dist-test-slave-wftq krb5kdc[14195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709028, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:28.169839 14301 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:10:28.170204 14301 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:28.170256 14301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:28.170403 14301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:28.171578 14301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "d2e3163f9dc741b590f003780afc9f1c"
format_stamp: "Formatted at 2026-09-29 19:10:28 on dist-test-slave-wftq"
server_key: "8aa903deb61fdbb5409cc734a41e9cd3"
server_key_iv: "1e39e5196e5bb7349c456c122ccd9d09"
server_key_version: "encryptionkey@0"
I20260929 19:10:28.171937 14301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "d2e3163f9dc741b590f003780afc9f1c"
format_stamp: "Formatted at 2026-09-29 19:10:28 on dist-test-slave-wftq"
server_key: "8aa903deb61fdbb5409cc734a41e9cd3"
server_key_iv: "1e39e5196e5bb7349c456c122ccd9d09"
server_key_version: "encryptionkey@0"
I20260929 19:10:28.173259 14301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:10:28.174021 14315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:28.174219 14301 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:28.174302 14301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "d2e3163f9dc741b590f003780afc9f1c"
format_stamp: "Formatted at 2026-09-29 19:10:28 on dist-test-slave-wftq"
server_key: "8aa903deb61fdbb5409cc734a41e9cd3"
server_key_iv: "1e39e5196e5bb7349c456c122ccd9d09"
server_key_version: "encryptionkey@0"
I20260929 19:10:28.174365 14301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:28.197786 14301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:28.200116 14301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:28.200273 14301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:28.200533 14301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:28.200874 14301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:28.200915 14301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:28.200947 14301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:28.200971 14301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:28.207966 14301 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:43613
I20260929 19:10:28.208014 14428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:43613 every 8 connection(s)
I20260929 19:10:28.208300 14301 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:10:28.209892 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 14301
I20260929 19:10:28.210008 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:10:28.210215 10160 external_mini_cluster.cc:1428] Setting key a08329f49c35f19f6ab6ed1e8e34b6f9
Sep 29 19:10:28 dist-test-slave-wftq krb5kdc[14195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709028, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.236.2@KRBTEST.COM; defaulting to no policy
I20260929 19:10:28.218900 14278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:28.209079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:33979 (local address 127.9.236.62:34871)
0929 19:10:28.209190 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:28.209193 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:28.209385 (+ 192us) server_negotiation.cc:407] Connection header received
0929 19:10:28.209953 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:28.209955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:28.209968 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:28.210072 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:28.211311 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.211876 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.213004 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.213341 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.213403 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:28.215882 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:28.215883 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:28.215883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:28.215890 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:28.217473 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:28.217943 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:28.217944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:28.217944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:28.217995 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:28.218258 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:28.218258 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:28.218259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:28.218369 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:28.218418 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:28.218748 (+ 330us) server_negotiation.cc:299] Negotiation successful
0929 19:10:28.218823 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 19:10:28.219056 14431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:28.209020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:34871 (local address 127.9.236.1:33979)
0929 19:10:28.209214 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:28.209258 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:28.209834 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:28.210194 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:28.210198 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:28.210538 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:28.211217 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:28.211224 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.211978 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.211979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:28.212895 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:28.212898 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.213427 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.213428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:28.213564 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:28.214088 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:28.214097 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:28.215783 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:28.217573 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:28.217581 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:28.217583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:28.217863 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:28.218062 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:28.218063 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:28.218063 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:28.218192 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:28.218433 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:28.218439 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:28.218602 (+ 163us) client_negotiation.cc:769] Sending connection context
0929 19:10:28.218724 (+ 122us) client_negotiation.cc:240] Negotiation successful
0929 19:10:28.218859 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":110,"threads_started":1}
I20260929 19:10:28.219641 14429 heartbeater.cc:347] Connected to a master server at 127.9.236.62:34871
I20260929 19:10:28.219728 14429 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:28.219851 14429 heartbeater.cc:511] Master 127.9.236.62:34871 requested a full tablet report, sending...
I20260929 19:10:28.220400 14239 ts_manager.cc:194] Registered new tserver with Master: d2e3163f9dc741b590f003780afc9f1c (127.9.236.1:43613)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.2@KRBTEST.COM" created.
I20260929 19:10:28.221194 14239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:33979
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
I20260929 19:10:28.235368 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:34871
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:28.313803 14434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:28.314015 14434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:28.314038 14434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:28.315138 14434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:28.315177 14434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:28.315201 14434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:10:28.316648 14434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:34871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:28.316826 14434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:28.317000 14434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:10:28.322296 14434 server_base.cc:1034] running on GCE node
W20260929 19:10:28.322232 14442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:28.322247 14439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:28.322232 14440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:28.322769 14434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:28.323007 14434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:28.324153 14434 hybrid_clock.cc:648] HybridClock initialized: now 1790709028324140 us; error 39 us; skew 500 ppm
Sep 29 19:10:28 dist-test-slave-wftq krb5kdc[14195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709028, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:28.326686 14434 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:10:28.327023 14434 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:28.327075 14434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:28.327183 14434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:28.328270 14434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "3d3dd04486e54b80a7ceae9faffbdc82"
format_stamp: "Formatted at 2026-09-29 19:10:28 on dist-test-slave-wftq"
server_key: "a99ad4e2a4375a4b493a12946de38e50"
server_key_iv: "3cbe2ef1e69cfab9ac728eb2cc94c5eb"
server_key_version: "encryptionkey@0"
I20260929 19:10:28.328591 14434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "3d3dd04486e54b80a7ceae9faffbdc82"
format_stamp: "Formatted at 2026-09-29 19:10:28 on dist-test-slave-wftq"
server_key: "a99ad4e2a4375a4b493a12946de38e50"
server_key_iv: "3cbe2ef1e69cfab9ac728eb2cc94c5eb"
server_key_version: "encryptionkey@0"
I20260929 19:10:28.329923 14434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:10:28.330655 14448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:28.330834 14434 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:28.330916 14434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "3d3dd04486e54b80a7ceae9faffbdc82"
format_stamp: "Formatted at 2026-09-29 19:10:28 on dist-test-slave-wftq"
server_key: "a99ad4e2a4375a4b493a12946de38e50"
server_key_iv: "3cbe2ef1e69cfab9ac728eb2cc94c5eb"
server_key_version: "encryptionkey@0"
I20260929 19:10:28.331000 14434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:28.348150 14434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:28.350121 14434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:28.350232 14434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:28.350449 14434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:28.350731 14434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:28.350765 14434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:28.350801 14434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:28.350829 14434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:28.357012 14434 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:38555
I20260929 19:10:28.357062 14561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:38555 every 8 connection(s)
I20260929 19:10:28.357316 14434 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:10:28.360131 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 14434
I20260929 19:10:28.360245 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:10:28.360458 10160 external_mini_cluster.cc:1428] Setting key 83b0fec88e1d7061631038be47c9a47a
Sep 29 19:10:28 dist-test-slave-wftq krb5kdc[14195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709028, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:28.367300 14278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:28.358125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:57077 (local address 127.9.236.62:34871)
0929 19:10:28.358191 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:28.358193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:28.358425 (+ 232us) server_negotiation.cc:407] Connection header received
0929 19:10:28.358891 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:28.358892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:28.358904 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:28.358965 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:28.359809 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.360547 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.361733 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.362001 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.362078 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:28.364187 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:28.364190 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:28.364190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:28.364199 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:28.365471 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:28.366017 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:28.366018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:28.366019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:28.366100 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:28.366481 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:28.366482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:28.366482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:28.366715 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:28.366793 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:28.367156 (+ 363us) server_negotiation.cc:299] Negotiation successful
0929 19:10:28.367234 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 19:10:28.367616 14564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:28.358095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:34871 (local address 127.9.236.2:57077)
0929 19:10:28.358297 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:28.358335 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:28.358797 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:28.359028 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:28.359032 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:28.359265 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:28.359723 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:28.359727 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.360714 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.360715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:28.361649 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:28.361652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.362080 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.362080 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:28.362174 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:28.362618 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:28.362625 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:28.364085 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:28.365577 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:28.365586 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:28.365589 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:28.365934 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:28.366189 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:28.366190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:28.366191 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:28.366377 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:28.366794 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:28.366801 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:28.366998 (+ 197us) client_negotiation.cc:769] Sending connection context
0929 19:10:28.367146 (+ 148us) client_negotiation.cc:240] Negotiation successful
0929 19:10:28.367351 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":111,"threads_started":1}
I20260929 19:10:28.368117 14562 heartbeater.cc:347] Connected to a master server at 127.9.236.62:34871
I20260929 19:10:28.368211 14562 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:28.368346 14562 heartbeater.cc:511] Master 127.9.236.62:34871 requested a full tablet report, sending...
I20260929 19:10:28.368733 14239 ts_manager.cc:194] Registered new tserver with Master: 3d3dd04486e54b80a7ceae9faffbdc82 (127.9.236.2:38555)
I20260929 19:10:28.369132 14239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:57077
WARNING: no policy specified for kudu/127.9.236.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
I20260929 19:10:28.387825 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:34871
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:28.475401 14567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:28.475628 14567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:28.475652 14567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:28.476785 14567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:28.476820 14567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:28.476845 14567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:10:28.478295 14567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:34871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:28.478484 14567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:28.478681 14567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:28.483594 14572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:28.483615 14573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:28.483639 14575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:28.483994 14567 server_base.cc:1034] running on GCE node
I20260929 19:10:28.484220 14567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:28.484469 14567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:28.485611 14567 hybrid_clock.cc:648] HybridClock initialized: now 1790709028485593 us; error 34 us; skew 500 ppm
Sep 29 19:10:28 dist-test-slave-wftq krb5kdc[14195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709028, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:28.488089 14567 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:10:28.488420 14567 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:28.488469 14567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:28.488593 14567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:28.489708 14567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "f29dea36de3948e0942dd9f21363acb3"
format_stamp: "Formatted at 2026-09-29 19:10:28 on dist-test-slave-wftq"
server_key: "da83fcd2a2585ad47ea29da9d2435a6f"
server_key_iv: "71b7cef186bafb98fe9871b91af6f16a"
server_key_version: "encryptionkey@0"
I20260929 19:10:28.490041 14567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "f29dea36de3948e0942dd9f21363acb3"
format_stamp: "Formatted at 2026-09-29 19:10:28 on dist-test-slave-wftq"
server_key: "da83fcd2a2585ad47ea29da9d2435a6f"
server_key_iv: "71b7cef186bafb98fe9871b91af6f16a"
server_key_version: "encryptionkey@0"
I20260929 19:10:28.491277 14567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:10:28.491976 14581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:28.492184 14567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:28.492268 14567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "f29dea36de3948e0942dd9f21363acb3"
format_stamp: "Formatted at 2026-09-29 19:10:28 on dist-test-slave-wftq"
server_key: "da83fcd2a2585ad47ea29da9d2435a6f"
server_key_iv: "71b7cef186bafb98fe9871b91af6f16a"
server_key_version: "encryptionkey@0"
I20260929 19:10:28.492329 14567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:28.507766 14567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:28.509934 14567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:28.510067 14567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:28.510277 14567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:28.510573 14567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:28.510607 14567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:28.510635 14567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:28.510651 14567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:28.516901 14567 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:39379
I20260929 19:10:28.517004 14694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:39379 every 8 connection(s)
I20260929 19:10:28.517218 14567 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:10:28.523065 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 14567
I20260929 19:10:28.523195 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:10:28.523373 10160 external_mini_cluster.cc:1428] Setting key f0a9d6f8887270fe5488b783f8697045
Sep 29 19:10:28 dist-test-slave-wftq krb5kdc[14195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709028, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:28.527026 14278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:28.517956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:34843 (local address 127.9.236.62:34871)
0929 19:10:28.518018 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:28.518020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:28.518281 (+ 261us) server_negotiation.cc:407] Connection header received
0929 19:10:28.518879 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:28.518881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:28.518891 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:28.518950 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:28.519994 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.520505 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.521785 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.521998 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.522056 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:28.524731 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:28.524733 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:28.524733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:28.524739 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:28.525851 (+ 1112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:28.526251 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:28.526252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:28.526252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:28.526295 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:28.526500 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:28.526501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:28.526501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:28.526591 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:28.526639 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:28.526899 (+ 260us) server_negotiation.cc:299] Negotiation successful
0929 19:10:28.526954 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 19:10:28.527280 14697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:28.517951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:34871 (local address 127.9.236.3:34843)
0929 19:10:28.518131 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:28.518174 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:28.518781 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:28.519033 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:28.519037 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:28.519333 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:28.519891 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:28.519897 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.520648 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.520650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:28.521670 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:28.521673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.522071 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.522072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:28.522226 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:28.522811 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:28.522820 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:28.524654 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:28.525923 (+ 1269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:28.525929 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:28.525932 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:28.526181 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:28.526353 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:28.526354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:28.526354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:28.526448 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:28.526667 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:28.526671 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:28.526791 (+ 120us) client_negotiation.cc:769] Sending connection context
0929 19:10:28.526881 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 19:10:28.527014 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
I20260929 19:10:28.527793 14695 heartbeater.cc:347] Connected to a master server at 127.9.236.62:34871
I20260929 19:10:28.527897 14695 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:28.528069 14695 heartbeater.cc:511] Master 127.9.236.62:34871 requested a full tablet report, sending...
I20260929 19:10:28.528470 14239 ts_manager.cc:194] Registered new tserver with Master: f29dea36de3948e0942dd9f21363acb3 (127.9.236.3:39379)
I20260929 19:10:28.528828 14239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:34843
I20260929 19:10:28.535233 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:10:28.541733 14278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:28.536474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43196 (local address 127.9.236.62:34871)
0929 19:10:28.536529 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:28.536532 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:28.536646 (+ 114us) server_negotiation.cc:407] Connection header received
0929 19:10:28.536706 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:28.536706 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:28.536716 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:28.536755 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:28.537297 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.537777 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.538477 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.538663 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.538712 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:28.539743 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:28.539744 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:28.539744 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:28.539750 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:28.540738 (+ 988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:28.541026 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:28.541027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:28.541027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:28.541077 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:28.541264 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:28.541265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:28.541265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:28.541399 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:28.541495 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:28.541579 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 19:10:28.541650 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 19:10:28.543180 14239 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:10:28.545037 14239 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:10:28.551673 14704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:28.547445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:39379 (local address 127.0.0.1:53108)
0929 19:10:28.547744 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:28.547754 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:28.547811 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:28.548126 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:28.548129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:28.548132 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:28.548197 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:28.548199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.549279 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.549280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:28.550433 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:28.550435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.551356 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.551357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:28.551446 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:28.551460 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 19:10:28.551525 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 19:10:28.551589 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":69,"threads_started":1}
I20260929 19:10:28.551687 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:28.547747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53108 (local address 127.9.236.3:39379)
0929 19:10:28.547937 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:28.547942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:28.547951 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:28.547990 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:28.547992 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:28.548009 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:28.548092 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:28.548302 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.549202 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.550529 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.551278 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.551338 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:28.551402 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:28.551546 (+ 144us) server_negotiation.cc:299] Negotiation successful
0929 19:10:28.551594 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":118,"threads_started":1}
I20260929 19:10:28.551905 14703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:28.547394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:43613 (local address 127.0.0.1:46274)
0929 19:10:28.547588 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:28.547616 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:28.547679 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:28.548379 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:28.548380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:28.548382 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:28.548418 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:28.548419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.549777 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.549778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:28.550764 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:28.550766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.551658 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.551659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:28.551725 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:28.551730 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:10:28.551777 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 19:10:28.551824 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":9,"thread_start_us":58,"threads_started":1}
I20260929 19:10:28.552222 14705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:28.547522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46274 (local address 127.9.236.1:43613)
0929 19:10:28.548221 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:28.548225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:28.548234 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:28.548273 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:28.548275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:28.548289 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:28.548363 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:28.548505 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.549662 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.550849 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.551497 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.551567 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:28.551622 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:28.552096 (+ 474us) server_negotiation.cc:299] Negotiation successful
0929 19:10:28.552143 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":113,"threads_started":1}
I20260929 19:10:28.552574 14706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:28.547635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:38555 (local address 127.0.0.1:60506)
0929 19:10:28.548437 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:28.548445 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:28.548495 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:28.548731 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:28.548732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:28.548734 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:28.548778 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:28.548779 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.550028 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.550030 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:28.551082 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:28.551084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.552346 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.552347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:28.552401 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:28.552405 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:10:28.552450 (+ 45us) client_negotiation.cc:240] Negotiation successful
0929 19:10:28.552508 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":792,"thread_start_us":41,"threads_started":1}
I20260929 19:10:28.552590 14707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:28.547754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60506 (local address 127.9.236.2:38555)
0929 19:10:28.547953 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:28.547957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:28.548556 (+ 599us) server_negotiation.cc:407] Connection header received
0929 19:10:28.548620 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:28.548623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:28.548639 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:28.548716 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:28.548856 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.549921 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.551182 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.552197 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.552261 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:28.552310 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:28.552476 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 19:10:28.552515 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":112,"threads_started":1}
I20260929 19:10:28.552618 14629 tablet_service.cc:1467] Processing CreateTablet for tablet 5bdafb5357b74fb4b21bdd1e6d35610d (DEFAULT_TABLE table=test-table [id=04d76bc5d70c452b975f08ac928cc733]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:28.552824 14363 tablet_service.cc:1467] Processing CreateTablet for tablet 5bdafb5357b74fb4b21bdd1e6d35610d (DEFAULT_TABLE table=test-table [id=04d76bc5d70c452b975f08ac928cc733]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:28.553006 14629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bdafb5357b74fb4b21bdd1e6d35610d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:28.553092 14363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bdafb5357b74fb4b21bdd1e6d35610d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:28.553789 14496 tablet_service.cc:1467] Processing CreateTablet for tablet 5bdafb5357b74fb4b21bdd1e6d35610d (DEFAULT_TABLE table=test-table [id=04d76bc5d70c452b975f08ac928cc733]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:28.554091 14496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bdafb5357b74fb4b21bdd1e6d35610d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:28.556473 14709 tablet_bootstrap.cc:492] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82: Bootstrap starting.
I20260929 19:10:28.556752 14710 tablet_bootstrap.cc:492] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3: Bootstrap starting.
I20260929 19:10:28.557000 14711 tablet_bootstrap.cc:492] T 5bdafb5357b74fb4b21bdd1e6d35610d P d2e3163f9dc741b590f003780afc9f1c: Bootstrap starting.
I20260929 19:10:28.557066 14709 tablet_bootstrap.cc:654] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:28.557495 14710 tablet_bootstrap.cc:654] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:28.557569 14709 log.cc:824] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:28.557636 14711 tablet_bootstrap.cc:654] T 5bdafb5357b74fb4b21bdd1e6d35610d P d2e3163f9dc741b590f003780afc9f1c: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:28.558133 14710 log.cc:824] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:28.558171 14709 tablet_bootstrap.cc:492] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82: No bootstrap required, opened a new log
I20260929 19:10:28.558212 14711 log.cc:824] T 5bdafb5357b74fb4b21bdd1e6d35610d P d2e3163f9dc741b590f003780afc9f1c: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:28.558223 14709 ts_tablet_manager.cc:1397] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:10:28.558888 14711 tablet_bootstrap.cc:492] T 5bdafb5357b74fb4b21bdd1e6d35610d P d2e3163f9dc741b590f003780afc9f1c: No bootstrap required, opened a new log
I20260929 19:10:28.558949 14710 tablet_bootstrap.cc:492] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3: No bootstrap required, opened a new log
I20260929 19:10:28.558962 14711 ts_tablet_manager.cc:1397] T 5bdafb5357b74fb4b21bdd1e6d35610d P d2e3163f9dc741b590f003780afc9f1c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:10:28.558995 14710 ts_tablet_manager.cc:1397] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:10:28.559728 14709 raft_consensus.cc:348] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e3163f9dc741b590f003780afc9f1c" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43613 } } peers { permanent_uuid: "f29dea36de3948e0942dd9f21363acb3" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 39379 } } peers { permanent_uuid: "3d3dd04486e54b80a7ceae9faffbdc82" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 38555 } }
I20260929 19:10:28.559854 14709 raft_consensus.cc:374] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:28.559885 14709 raft_consensus.cc:729] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d3dd04486e54b80a7ceae9faffbdc82, State: Initialized, Role: FOLLOWER
I20260929 19:10:28.560015 14709 consensus_queue.cc:260] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e3163f9dc741b590f003780afc9f1c" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43613 } } peers { permanent_uuid: "f29dea36de3948e0942dd9f21363acb3" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 39379 } } peers { permanent_uuid: "3d3dd04486e54b80a7ceae9faffbdc82" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 38555 } }
I20260929 19:10:28.560249 14709 ts_tablet_manager.cc:1428] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:10:28.560364 14562 heartbeater.cc:503] Master 127.9.236.62:34871 was elected leader, sending a full tablet report...
I20260929 19:10:28.560978 14710 raft_consensus.cc:348] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e3163f9dc741b590f003780afc9f1c" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43613 } } peers { permanent_uuid: "f29dea36de3948e0942dd9f21363acb3" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 39379 } } peers { permanent_uuid: "3d3dd04486e54b80a7ceae9faffbdc82" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 38555 } }
I20260929 19:10:28.561190 14710 raft_consensus.cc:374] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:28.561236 14710 raft_consensus.cc:729] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f29dea36de3948e0942dd9f21363acb3, State: Initialized, Role: FOLLOWER
I20260929 19:10:28.561285 14711 raft_consensus.cc:348] T 5bdafb5357b74fb4b21bdd1e6d35610d P d2e3163f9dc741b590f003780afc9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e3163f9dc741b590f003780afc9f1c" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43613 } } peers { permanent_uuid: "f29dea36de3948e0942dd9f21363acb3" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 39379 } } peers { permanent_uuid: "3d3dd04486e54b80a7ceae9faffbdc82" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 38555 } }
I20260929 19:10:28.561367 14710 consensus_queue.cc:260] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e3163f9dc741b590f003780afc9f1c" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43613 } } peers { permanent_uuid: "f29dea36de3948e0942dd9f21363acb3" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 39379 } } peers { permanent_uuid: "3d3dd04486e54b80a7ceae9faffbdc82" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 38555 } }
I20260929 19:10:28.561416 14711 raft_consensus.cc:374] T 5bdafb5357b74fb4b21bdd1e6d35610d P d2e3163f9dc741b590f003780afc9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:28.561447 14711 raft_consensus.cc:729] T 5bdafb5357b74fb4b21bdd1e6d35610d P d2e3163f9dc741b590f003780afc9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e3163f9dc741b590f003780afc9f1c, State: Initialized, Role: FOLLOWER
I20260929 19:10:28.561547 14711 consensus_queue.cc:260] T 5bdafb5357b74fb4b21bdd1e6d35610d P d2e3163f9dc741b590f003780afc9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e3163f9dc741b590f003780afc9f1c" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43613 } } peers { permanent_uuid: "f29dea36de3948e0942dd9f21363acb3" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 39379 } } peers { permanent_uuid: "3d3dd04486e54b80a7ceae9faffbdc82" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 38555 } }
I20260929 19:10:28.561658 14710 ts_tablet_manager.cc:1428] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260929 19:10:28.561676 14695 heartbeater.cc:503] Master 127.9.236.62:34871 was elected leader, sending a full tablet report...
I20260929 19:10:28.561932 14711 ts_tablet_manager.cc:1428] T 5bdafb5357b74fb4b21bdd1e6d35610d P d2e3163f9dc741b590f003780afc9f1c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:10:28.561985 14429 heartbeater.cc:503] Master 127.9.236.62:34871 was elected leader, sending a full tablet report...
W20260929 19:10:28.607806 14563 tablet.cc:2369] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 19:10:28.709026 14430 tablet.cc:2369] T 5bdafb5357b74fb4b21bdd1e6d35610d P d2e3163f9dc741b590f003780afc9f1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:10:28.716307 14716 raft_consensus.cc:482] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:10:28.716457 14716 raft_consensus.cc:504] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e3163f9dc741b590f003780afc9f1c" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43613 } } peers { permanent_uuid: "f29dea36de3948e0942dd9f21363acb3" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 39379 } } peers { permanent_uuid: "3d3dd04486e54b80a7ceae9faffbdc82" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 38555 } }
I20260929 19:10:28.716840 14716 leader_election.cc:288] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2e3163f9dc741b590f003780afc9f1c (127.9.236.1:43613), 3d3dd04486e54b80a7ceae9faffbdc82 (127.9.236.2:38555)
I20260929 19:10:28.720244 14697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:28.717133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:43613 (local address 127.9.236.3:49329)
0929 19:10:28.717204 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:28.717217 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:28.717284 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:28.717506 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:28.717507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:28.717509 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:28.717604 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:28.717607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.718362 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.718363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:28.719301 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:28.719304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.720004 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.720005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:28.720083 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:28.720088 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:10:28.720131 (+ 43us) client_negotiation.cc:240] Negotiation successful
0929 19:10:28.720171 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260929 19:10:28.720290 14705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:28.717148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:49329 (local address 127.9.236.1:43613)
0929 19:10:28.717204 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:28.717206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:28.717277 (+ 71us) server_negotiation.cc:407] Connection header received
0929 19:10:28.717378 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:28.717380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:28.717392 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:28.717487 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:28.717720 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.718273 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.719397 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.719923 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.719999 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:28.720013 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:28.720155 (+ 142us) server_negotiation.cc:299] Negotiation successful
0929 19:10:28.720202 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 19:10:28.720546 14707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:28.717213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:37529 (local address 127.9.236.2:38555)
0929 19:10:28.717258 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:28.717260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:28.717496 (+ 236us) server_negotiation.cc:407] Connection header received
0929 19:10:28.717535 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:28.717537 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:28.717552 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:28.717659 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:28.717912 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.718461 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.719606 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.720152 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.720219 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:28.720230 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:28.720412 (+ 182us) server_negotiation.cc:299] Negotiation successful
0929 19:10:28.720486 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:10:28.720763 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:28.717134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:38555 (local address 127.9.236.3:37529)
0929 19:10:28.717386 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:28.717401 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:28.717481 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:28.717723 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:28.717724 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:28.717727 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:28.717816 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:28.717819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.718549 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.718550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:28.719515 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:28.719518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.720239 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.720240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:28.720299 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:28.720303 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:10:28.720647 (+ 344us) client_negotiation.cc:240] Negotiation successful
0929 19:10:28.720695 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":17,"thread_start_us":122,"threads_started":1}
I20260929 19:10:28.720922 14383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bdafb5357b74fb4b21bdd1e6d35610d" candidate_uuid: "f29dea36de3948e0942dd9f21363acb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e3163f9dc741b590f003780afc9f1c" is_pre_election: true
I20260929 19:10:28.721087 14383 raft_consensus.cc:2454] T 5bdafb5357b74fb4b21bdd1e6d35610d P d2e3163f9dc741b590f003780afc9f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f29dea36de3948e0942dd9f21363acb3 in term 0.
I20260929 19:10:28.721153 14516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bdafb5357b74fb4b21bdd1e6d35610d" candidate_uuid: "f29dea36de3948e0942dd9f21363acb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d3dd04486e54b80a7ceae9faffbdc82" is_pre_election: true
I20260929 19:10:28.721303 14516 raft_consensus.cc:2454] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f29dea36de3948e0942dd9f21363acb3 in term 0.
I20260929 19:10:28.721400 14585 leader_election.cc:302] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2e3163f9dc741b590f003780afc9f1c, f29dea36de3948e0942dd9f21363acb3; no voters:
I20260929 19:10:28.721560 14716 raft_consensus.cc:2781] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:10:28.721647 14716 raft_consensus.cc:482] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:10:28.721671 14716 raft_consensus.cc:3037] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:28.722637 14716 raft_consensus.cc:504] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e3163f9dc741b590f003780afc9f1c" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43613 } } peers { permanent_uuid: "f29dea36de3948e0942dd9f21363acb3" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 39379 } } peers { permanent_uuid: "3d3dd04486e54b80a7ceae9faffbdc82" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 38555 } }
I20260929 19:10:28.722776 14716 leader_election.cc:288] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 [CANDIDATE]: Term 1 election: Requested vote from peers d2e3163f9dc741b590f003780afc9f1c (127.9.236.1:43613), 3d3dd04486e54b80a7ceae9faffbdc82 (127.9.236.2:38555)
I20260929 19:10:28.722949 14383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bdafb5357b74fb4b21bdd1e6d35610d" candidate_uuid: "f29dea36de3948e0942dd9f21363acb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e3163f9dc741b590f003780afc9f1c"
I20260929 19:10:28.722954 14516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bdafb5357b74fb4b21bdd1e6d35610d" candidate_uuid: "f29dea36de3948e0942dd9f21363acb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d3dd04486e54b80a7ceae9faffbdc82"
I20260929 19:10:28.723026 14383 raft_consensus.cc:3037] T 5bdafb5357b74fb4b21bdd1e6d35610d P d2e3163f9dc741b590f003780afc9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:28.723037 14516 raft_consensus.cc:3037] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:28.723789 14516 raft_consensus.cc:2454] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f29dea36de3948e0942dd9f21363acb3 in term 1.
I20260929 19:10:28.723824 14383 raft_consensus.cc:2454] T 5bdafb5357b74fb4b21bdd1e6d35610d P d2e3163f9dc741b590f003780afc9f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f29dea36de3948e0942dd9f21363acb3 in term 1.
I20260929 19:10:28.723980 14583 leader_election.cc:302] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d3dd04486e54b80a7ceae9faffbdc82, f29dea36de3948e0942dd9f21363acb3; no voters:
I20260929 19:10:28.724084 14716 raft_consensus.cc:2781] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:28.724273 14716 raft_consensus.cc:686] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 [term 1 LEADER]: Becoming Leader. State: Replica: f29dea36de3948e0942dd9f21363acb3, State: Running, Role: LEADER
I20260929 19:10:28.724375 14716 consensus_queue.cc:237] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e3163f9dc741b590f003780afc9f1c" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43613 } } peers { permanent_uuid: "f29dea36de3948e0942dd9f21363acb3" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 39379 } } peers { permanent_uuid: "3d3dd04486e54b80a7ceae9faffbdc82" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 38555 } }
I20260929 19:10:28.725404 14239 catalog_manager.cc:5495] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to f29dea36de3948e0942dd9f21363acb3 (127.9.236.3). New cstate: current_term: 1 leader_uuid: "f29dea36de3948e0942dd9f21363acb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e3163f9dc741b590f003780afc9f1c" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f29dea36de3948e0942dd9f21363acb3" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 39379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d3dd04486e54b80a7ceae9faffbdc82" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 38555 } health_report { overall_health: UNKNOWN } } }
I20260929 19:10:28.735656 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:28.733445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53114 (local address 127.9.236.3:39379)
0929 19:10:28.733494 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:28.733496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:28.733503 (+ 7us) server_negotiation.cc:407] Connection header received
0929 19:10:28.733537 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:28.733538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:28.733559 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:28.733619 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:28.733746 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.734232 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.734943 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:28.735145 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:28.735198 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:28.735322 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:28.735431 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:28.735499 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:28.735519 (+ 20us) server_negotiation.cc:299] Negotiation successful
0929 19:10:28.735536 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 19:10:28.737571 14516 raft_consensus.cc:1261] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82 [term 1 FOLLOWER]: Refusing update from remote peer f29dea36de3948e0942dd9f21363acb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:28.737610 14383 raft_consensus.cc:1261] T 5bdafb5357b74fb4b21bdd1e6d35610d P d2e3163f9dc741b590f003780afc9f1c [term 1 FOLLOWER]: Refusing update from remote peer f29dea36de3948e0942dd9f21363acb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:28.737910 14716 consensus_queue.cc:1035] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e3163f9dc741b590f003780afc9f1c" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:28.738075 14719 consensus_queue.cc:1035] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d3dd04486e54b80a7ceae9faffbdc82" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 38555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:28.740697 14722 mvcc.cc:205] Tried to move back new op lower bound from 7334744181706354688 to 7334744181655900160. Current Snapshot: MvccSnapshot[applied={T|T < 7334744181706354688}]
I20260929 19:10:28.741189 14723 mvcc.cc:205] Tried to move back new op lower bound from 7334744181706354688 to 7334744181655900160. Current Snapshot: MvccSnapshot[applied={T|T < 7334744181706354688}]
I20260929 19:10:28.741904 14724 mvcc.cc:205] Tried to move back new op lower bound from 7334744181706354688 to 7334744181655900160. Current Snapshot: MvccSnapshot[applied={T|T < 7334744181706354688 or (T in {7334744181706354688})}]
I20260929 19:10:28.743276 14238 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:28.743386 14238 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:28.744206 14238 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f352c2be10a4caea1ff403a6abe8aad: Sending DeleteTablet for 3 replicas of tablet 5bdafb5357b74fb4b21bdd1e6d35610d
I20260929 19:10:28.744519 14363 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bdafb5357b74fb4b21bdd1e6d35610d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:28 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:46274
I20260929 19:10:28.744654 14629 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bdafb5357b74fb4b21bdd1e6d35610d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:28 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:53108
I20260929 19:10:28.744693 14496 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bdafb5357b74fb4b21bdd1e6d35610d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:28 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:60506
I20260929 19:10:28.744871 14732 tablet_replica.cc:330] T 5bdafb5357b74fb4b21bdd1e6d35610d P d2e3163f9dc741b590f003780afc9f1c: stopping tablet replica
I20260929 19:10:28.744946 14734 tablet_replica.cc:330] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82: stopping tablet replica
I20260929 19:10:28.745013 14733 tablet_replica.cc:330] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3: stopping tablet replica
I20260929 19:10:28.745056 14734 raft_consensus.cc:2229] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:28.745132 14733 raft_consensus.cc:2229] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:10:28.745186 14734 raft_consensus.cc:2258] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:28.745366 14733 raft_consensus.cc:2258] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:28.745385 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 14301
I20260929 19:10:28.746958 14733 ts_tablet_manager.cc:1905] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:28.748879 14733 ts_tablet_manager.cc:1918] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:10:28.748944 14733 log.cc:1196] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/5bdafb5357b74fb4b21bdd1e6d35610d
I20260929 19:10:28.749243 14733 ts_tablet_manager.cc:1939] T 5bdafb5357b74fb4b21bdd1e6d35610d P f29dea36de3948e0942dd9f21363acb3: Deleting consensus metadata
I20260929 19:10:28.749347 14734 ts_tablet_manager.cc:1905] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:28.749650 14224 catalog_manager.cc:4841] TS f29dea36de3948e0942dd9f21363acb3 (127.9.236.3:39379): tablet 5bdafb5357b74fb4b21bdd1e6d35610d (table test-table [id=04d76bc5d70c452b975f08ac928cc733]) successfully deleted
I20260929 19:10:28.751085 14734 ts_tablet_manager.cc:1918] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:10:28.751168 14734 log.cc:1196] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/5bdafb5357b74fb4b21bdd1e6d35610d
I20260929 19:10:28.751421 14734 ts_tablet_manager.cc:1939] T 5bdafb5357b74fb4b21bdd1e6d35610d P 3d3dd04486e54b80a7ceae9faffbdc82: Deleting consensus metadata
I20260929 19:10:28.751888 14224 catalog_manager.cc:4841] TS 3d3dd04486e54b80a7ceae9faffbdc82 (127.9.236.2:38555): tablet 5bdafb5357b74fb4b21bdd1e6d35610d (table test-table [id=04d76bc5d70c452b975f08ac928cc733]) successfully deleted
W20260929 19:10:28.752485 14226 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.236.1:43613 (error 108)
I20260929 19:10:28.752791 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 14434
I20260929 19:10:28.753430 14706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:28.752923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:43613 (local address 127.0.0.1:46288)
0929 19:10:28.753356 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:28.753394 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.1:43613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410}
W20260929 19:10:28.753605 14226 catalog_manager.cc:4584] TS d2e3163f9dc741b590f003780afc9f1c (127.9.236.1:43613): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5bdafb5357b74fb4b21bdd1e6d35610d: Network error: Client connection negotiation failed: client connection to 127.9.236.1:43613: connect: Connection refused (error 111)
I20260929 19:10:28.760615 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 14567
I20260929 19:10:28.765542 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 14208
2026-09-29T19:10:28Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1950 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-29T19:10:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:10:28Z Disabled control of system clock
I20260929 19:10:28.793366 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:46309
--webserver_interface=127.9.236.62
--webserver_port=0
--builtin_ntp_servers=127.9.236.20:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:46309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 19:10:28.874477 14739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:28.874676 14739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:28.874696 14739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:28.875738 14739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:28.875777 14739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:28.875788 14739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:28.875798 14739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:28.875809 14739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:28.877076 14739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:46309
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:46309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:28.877348 14739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:28.877622 14739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:28.882423 14745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:28.882405 14747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:28.882400 14744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:28.882716 14739 server_base.cc:1034] running on GCE node
I20260929 19:10:28.882980 14739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:28.883285 14739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:28.884424 14739 hybrid_clock.cc:648] HybridClock initialized: now 1790709028884402 us; error 38 us; skew 500 ppm
I20260929 19:10:28.885857 14739 webserver.cc:460] Webserver started at http://127.9.236.62:41783/ using document root <none> and password file <none>
I20260929 19:10:28.886142 14739 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:28.886196 14739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:28.886371 14739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:28.887578 14739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "9b4caf61ff6249a187840d6da32ef210"
format_stamp: "Formatted at 2026-09-29 19:10:28 on dist-test-slave-wftq"
server_key: "9902e0f78c1836ff39f418c587082b55"
server_key_iv: "a6f5e4aa65d1fefa8a824f44da40c0b1"
server_key_version: "encryptionkey@0"
I20260929 19:10:28.887953 14739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "9b4caf61ff6249a187840d6da32ef210"
format_stamp: "Formatted at 2026-09-29 19:10:28 on dist-test-slave-wftq"
server_key: "9902e0f78c1836ff39f418c587082b55"
server_key_iv: "a6f5e4aa65d1fefa8a824f44da40c0b1"
server_key_version: "encryptionkey@0"
I20260929 19:10:28.889431 14739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:10:28.890241 14753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:28.890429 14739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:28.890522 14739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "9b4caf61ff6249a187840d6da32ef210"
format_stamp: "Formatted at 2026-09-29 19:10:28 on dist-test-slave-wftq"
server_key: "9902e0f78c1836ff39f418c587082b55"
server_key_iv: "a6f5e4aa65d1fefa8a824f44da40c0b1"
server_key_version: "encryptionkey@0"
I20260929 19:10:28.890610 14739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:28.894397 14739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:28.894678 14739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:28.894797 14739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:28.899065 14739 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:46309
I20260929 19:10:28.899120 14805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:46309 every 8 connection(s)
I20260929 19:10:28.899564 14739 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:10:28.900279 14806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:28.903052 14806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210: Bootstrap starting.
I20260929 19:10:28.903568 14806 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:28.903796 14806 log.cc:824] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:28.904533 14806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210: No bootstrap required, opened a new log
I20260929 19:10:28.906123 14806 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b4caf61ff6249a187840d6da32ef210" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46309 } }
I20260929 19:10:28.906277 14806 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:28.906297 14806 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b4caf61ff6249a187840d6da32ef210, State: Initialized, Role: FOLLOWER
I20260929 19:10:28.906447 14806 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b4caf61ff6249a187840d6da32ef210" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46309 } }
I20260929 19:10:28.906525 14806 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:10:28.906546 14806 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:10:28.906571 14806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:28.907056 14806 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b4caf61ff6249a187840d6da32ef210" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46309 } }
I20260929 19:10:28.907140 14806 leader_election.cc:302] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b4caf61ff6249a187840d6da32ef210; no voters:
I20260929 19:10:28.907263 14806 leader_election.cc:288] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:10:28.907337 14809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:28.907514 14806 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:28.907519 14809 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210 [term 1 LEADER]: Becoming Leader. State: Replica: 9b4caf61ff6249a187840d6da32ef210, State: Running, Role: LEADER
I20260929 19:10:28.907666 14809 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b4caf61ff6249a187840d6da32ef210" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46309 } }
I20260929 19:10:28.908210 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 14739
I20260929 19:10:28.908311 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:10:28.908530 10160 external_mini_cluster.cc:1428] Setting key b328cadda6321cd513de32efad22017f
I20260929 19:10:28.909019 14811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b4caf61ff6249a187840d6da32ef210. Latest consensus state: current_term: 1 leader_uuid: "9b4caf61ff6249a187840d6da32ef210" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b4caf61ff6249a187840d6da32ef210" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46309 } } }
I20260929 19:10:28.909457 14810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b4caf61ff6249a187840d6da32ef210" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b4caf61ff6249a187840d6da32ef210" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46309 } } }
I20260929 19:10:28.909778 14811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:28.909809 14810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210 [sys.catalog]: This master's current role is: LEADER
W20260929 19:10:28.909627 14824 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:10:28.910048 14824 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:10:28.910172 14829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:28.910369 14829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:10:28.911526 14827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:28.909216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44706 (local address 127.9.236.62:46309)
0929 19:10:28.909423 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:28.909433 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:28.909748 (+ 315us) server_negotiation.cc:407] Connection header received
0929 19:10:28.910148 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:28.910156 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:28.910204 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:28.910371 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:28.911040 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:28.911043 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:28.911043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:10:28.911047 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:28.911086 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:10:28.911099 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:28.911164 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:28.911273 (+ 109us) server_negotiation.cc:299] Negotiation successful
0929 19:10:28.911336 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260929 19:10:28.912117 14829 catalog_manager.cc:1347] Generated new cluster ID: 39ed995b32564cf1afbd54aab364d672
I20260929 19:10:28.912166 14829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:10:28.951772 14829 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:10:28.951987 14829 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:28.956032 14829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210: Generated new TSK 0
I20260929 19:10:28.956192 14829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:10:28.962169 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:46309
--builtin_ntp_servers=127.9.236.20:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 19:10:29.051167 14831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:29.051404 14831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:29.051425 14831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:29.052599 14831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:29.052634 14831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:29.052660 14831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:10:29.054040 14831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:46309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:29.054263 14831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:29.054551 14831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:29.059700 14837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:29.059775 14836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:29.059819 14831 server_base.cc:1034] running on GCE node
W20260929 19:10:29.059762 14839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:29.060137 14831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:29.060427 14831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:29.061585 14831 hybrid_clock.cc:648] HybridClock initialized: now 1790709029061581 us; error 80 us; skew 500 ppm
I20260929 19:10:29.062911 14831 webserver.cc:460] Webserver started at http://127.9.236.1:38003/ using document root <none> and password file <none>
I20260929 19:10:29.063141 14831 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:29.063191 14831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:29.063320 14831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:29.064409 14831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "944fc11643044c3995e256b8fc65d464"
format_stamp: "Formatted at 2026-09-29 19:10:29 on dist-test-slave-wftq"
server_key: "2f11d5a15f466d00d5f394fa55eab0da"
server_key_iv: "beb09729a05736a2068acde477070931"
server_key_version: "encryptionkey@0"
I20260929 19:10:29.064743 14831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "944fc11643044c3995e256b8fc65d464"
format_stamp: "Formatted at 2026-09-29 19:10:29 on dist-test-slave-wftq"
server_key: "2f11d5a15f466d00d5f394fa55eab0da"
server_key_iv: "beb09729a05736a2068acde477070931"
server_key_version: "encryptionkey@0"
I20260929 19:10:29.066082 14831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:10:29.066826 14845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:29.067016 14831 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:10:29.067104 14831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "944fc11643044c3995e256b8fc65d464"
format_stamp: "Formatted at 2026-09-29 19:10:29 on dist-test-slave-wftq"
server_key: "2f11d5a15f466d00d5f394fa55eab0da"
server_key_iv: "beb09729a05736a2068acde477070931"
server_key_version: "encryptionkey@0"
I20260929 19:10:29.067164 14831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:29.070622 14831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:29.070902 14831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:29.071019 14831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:29.071202 14831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:29.071489 14831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:29.071528 14831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:29.071554 14831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:29.071571 14831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:29.077659 14831 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:44109
I20260929 19:10:29.077754 14958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:44109 every 8 connection(s)
I20260929 19:10:29.078052 14831 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:10:29.080988 14827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.078817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:60213 (local address 127.9.236.62:46309)
0929 19:10:29.078910 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:29.078912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:29.079076 (+ 164us) server_negotiation.cc:407] Connection header received
0929 19:10:29.079544 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:29.079545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:29.079554 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:29.079664 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:29.080760 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:29.080761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:29.080761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:10:29.080765 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:29.080783 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:10:29.080788 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:29.080841 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:29.080906 (+ 65us) server_negotiation.cc:299] Negotiation successful
0929 19:10:29.080924 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 19:10:29.081140 14961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.078749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46309 (local address 127.9.236.1:60213)
0929 19:10:29.078943 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:29.078982 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:29.079455 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:29.079726 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:29.079729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:29.079736 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:29.080619 (+ 883us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:10:29.080619 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:29.080646 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:10:29.080673 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:29.080841 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:29.080844 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:29.080846 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:10:29.080895 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 19:10:29.080940 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":102,"threads_started":1}
I20260929 19:10:29.081625 14959 heartbeater.cc:347] Connected to a master server at 127.9.236.62:46309
I20260929 19:10:29.081715 14959 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:29.081852 14959 heartbeater.cc:511] Master 127.9.236.62:46309 requested a full tablet report, sending...
I20260929 19:10:29.082348 14770 ts_manager.cc:194] Registered new tserver with Master: 944fc11643044c3995e256b8fc65d464 (127.9.236.1:44109)
I20260929 19:10:29.088217 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 14831
I20260929 19:10:29.088347 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:10:29.088513 10160 external_mini_cluster.cc:1428] Setting key 053bff8b756c472affd9bed07fc09af0
I20260929 19:10:29.089376 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:46309
--builtin_ntp_servers=127.9.236.20:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 19:10:29.177508 14962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:29.177747 14962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:29.177772 14962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:29.178937 14962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:29.178982 14962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:29.179009 14962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:10:29.180322 14962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:46309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:29.180532 14962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:29.180810 14962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:29.186106 14970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:29.186254 14967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:29.186142 14968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:29.186383 14962 server_base.cc:1034] running on GCE node
I20260929 19:10:29.186592 14962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:29.186846 14962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:29.187987 14962 hybrid_clock.cc:648] HybridClock initialized: now 1790709029187969 us; error 33 us; skew 500 ppm
I20260929 19:10:29.189430 14962 webserver.cc:460] Webserver started at http://127.9.236.2:34233/ using document root <none> and password file <none>
I20260929 19:10:29.189699 14962 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:29.189756 14962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:29.189895 14962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:29.191090 14962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "fd1e995782f84d5db04509225f1a173d"
format_stamp: "Formatted at 2026-09-29 19:10:29 on dist-test-slave-wftq"
server_key: "f7ca1d1c569e8a2bbe35c44c0e7e0d68"
server_key_iv: "af3e577493db0c42fa5b3225e511ddc0"
server_key_version: "encryptionkey@0"
I20260929 19:10:29.191458 14962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "fd1e995782f84d5db04509225f1a173d"
format_stamp: "Formatted at 2026-09-29 19:10:29 on dist-test-slave-wftq"
server_key: "f7ca1d1c569e8a2bbe35c44c0e7e0d68"
server_key_iv: "af3e577493db0c42fa5b3225e511ddc0"
server_key_version: "encryptionkey@0"
I20260929 19:10:29.192929 14962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:10:29.193859 14976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:29.194027 14962 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:29.194116 14962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "fd1e995782f84d5db04509225f1a173d"
format_stamp: "Formatted at 2026-09-29 19:10:29 on dist-test-slave-wftq"
server_key: "f7ca1d1c569e8a2bbe35c44c0e7e0d68"
server_key_iv: "af3e577493db0c42fa5b3225e511ddc0"
server_key_version: "encryptionkey@0"
I20260929 19:10:29.194177 14962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:29.197731 14962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:29.197980 14962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:29.198079 14962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:29.198254 14962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:29.198518 14962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:29.198547 14962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:29.198576 14962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:29.198596 14962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:29.205466 14962 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:32969
I20260929 19:10:29.205588 15089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:32969 every 8 connection(s)
I20260929 19:10:29.205881 14962 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:10:29.209193 14827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.206720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:55089 (local address 127.9.236.62:46309)
0929 19:10:29.206782 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:29.206784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:29.207029 (+ 245us) server_negotiation.cc:407] Connection header received
0929 19:10:29.207584 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:29.207586 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:29.207592 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:29.207644 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:29.208931 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:29.208932 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:29.208932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:10:29.208936 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:29.208953 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:10:29.208957 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:29.209016 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:29.209089 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 19:10:29.209122 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 19:10:29.209401 15092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.206707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46309 (local address 127.9.236.2:55089)
0929 19:10:29.206898 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:29.206936 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:29.207483 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:29.207797 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:29.207802 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:29.207812 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:29.208784 (+ 972us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:10:29.208785 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:29.208817 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:10:29.208845 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:29.209018 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:29.209024 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:29.209027 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:10:29.209117 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 19:10:29.209180 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260929 19:10:29.210084 15090 heartbeater.cc:347] Connected to a master server at 127.9.236.62:46309
I20260929 19:10:29.210171 15090 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:29.210294 15090 heartbeater.cc:511] Master 127.9.236.62:46309 requested a full tablet report, sending...
I20260929 19:10:29.210716 14770 ts_manager.cc:194] Registered new tserver with Master: fd1e995782f84d5db04509225f1a173d (127.9.236.2:32969)
I20260929 19:10:29.214162 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 14962
I20260929 19:10:29.214252 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:10:29.214392 10160 external_mini_cluster.cc:1428] Setting key dde037367cb4a001941fee6624542742
I20260929 19:10:29.215435 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:46309
--builtin_ntp_servers=127.9.236.20:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 19:10:29.297613 15093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:29.297833 15093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:29.297855 15093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:29.299009 15093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:29.299054 15093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:29.299079 15093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:10:29.300474 15093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:46309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:29.300671 15093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:29.300907 15093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:29.305814 15101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:29.305876 15098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:29.305828 15099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:29.306157 15093 server_base.cc:1034] running on GCE node
I20260929 19:10:29.306393 15093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:29.306632 15093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:29.307767 15093 hybrid_clock.cc:648] HybridClock initialized: now 1790709029307748 us; error 36 us; skew 500 ppm
I20260929 19:10:29.309057 15093 webserver.cc:460] Webserver started at http://127.9.236.3:39391/ using document root <none> and password file <none>
I20260929 19:10:29.309290 15093 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:29.309347 15093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:29.309439 15093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:29.310457 15093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "cc409ffbf92f48a69f51579e576a7881"
format_stamp: "Formatted at 2026-09-29 19:10:29 on dist-test-slave-wftq"
server_key: "648f6a464203708a38c7c067cd498b33"
server_key_iv: "1501c563c728377133b4739490359546"
server_key_version: "encryptionkey@0"
I20260929 19:10:29.310745 15093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "cc409ffbf92f48a69f51579e576a7881"
format_stamp: "Formatted at 2026-09-29 19:10:29 on dist-test-slave-wftq"
server_key: "648f6a464203708a38c7c067cd498b33"
server_key_iv: "1501c563c728377133b4739490359546"
server_key_version: "encryptionkey@0"
I20260929 19:10:29.311975 15093 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:10:29.312734 15107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:29.312939 15093 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:29.313021 15093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "cc409ffbf92f48a69f51579e576a7881"
format_stamp: "Formatted at 2026-09-29 19:10:29 on dist-test-slave-wftq"
server_key: "648f6a464203708a38c7c067cd498b33"
server_key_iv: "1501c563c728377133b4739490359546"
server_key_version: "encryptionkey@0"
I20260929 19:10:29.313095 15093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:29.316319 15093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:29.316553 15093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:29.316654 15093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:29.316854 15093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:29.317178 15093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:29.317217 15093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:29.317242 15093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:29.317268 15093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:29.323312 15093 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:41759
I20260929 19:10:29.323374 15220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:41759 every 8 connection(s)
I20260929 19:10:29.323642 15093 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:10:29.326547 14827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.324403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:52743 (local address 127.9.236.62:46309)
0929 19:10:29.324470 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:29.324472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:29.324624 (+ 152us) server_negotiation.cc:407] Connection header received
0929 19:10:29.325069 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:29.325070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:29.325082 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:29.325187 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:29.326324 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:29.326326 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:29.326326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:10:29.326330 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:29.326347 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:10:29.326351 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:29.326405 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:29.326468 (+ 63us) server_negotiation.cc:299] Negotiation successful
0929 19:10:29.326481 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 19:10:29.326676 15223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.324313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46309 (local address 127.9.236.3:52743)
0929 19:10:29.324499 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:29.324538 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:29.324981 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:29.325244 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:29.325248 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:29.325255 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:29.326163 (+ 908us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:10:29.326164 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:29.326190 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:10:29.326217 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:29.326412 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:29.326414 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:29.326417 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:10:29.326472 (+ 55us) client_negotiation.cc:240] Negotiation successful
0929 19:10:29.326516 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260929 19:10:29.327232 15221 heartbeater.cc:347] Connected to a master server at 127.9.236.62:46309
I20260929 19:10:29.327313 15221 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:29.327512 15221 heartbeater.cc:511] Master 127.9.236.62:46309 requested a full tablet report, sending...
I20260929 19:10:29.327939 14770 ts_manager.cc:194] Registered new tserver with Master: cc409ffbf92f48a69f51579e576a7881 (127.9.236.3:41759)
I20260929 19:10:29.330154 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 15093
I20260929 19:10:29.330266 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:10:29.330511 10160 external_mini_cluster.cc:1428] Setting key 4ea5406c68295aa012edea4de763a119
I20260929 19:10:29.331633 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:10:29.334725 14827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.333407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44720 (local address 127.9.236.62:46309)
0929 19:10:29.333450 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:29.333452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:29.333627 (+ 175us) server_negotiation.cc:407] Connection header received
0929 19:10:29.333690 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:29.333691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:29.333696 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:29.333759 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:29.334455 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:29.334456 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:29.334457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:10:29.334462 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:29.334481 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:10:29.334486 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:29.334514 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:29.334615 (+ 101us) server_negotiation.cc:299] Negotiation successful
0929 19:10:29.334632 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 19:10:29.336575 14770 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:10:29.338454 14770 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:10:29.342609 15230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.340833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:44109 (local address 127.0.0.1:50728)
0929 19:10:29.341292 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:29.341301 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:29.341341 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:29.341629 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:29.341635 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:29.341643 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:29.342152 (+ 509us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:10:29.342152 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:29.342166 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:10:29.342185 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:29.342430 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:29.342434 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:29.342437 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:10:29.342512 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 19:10:29.342526 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":81,"threads_started":1}
I20260929 19:10:29.342633 15233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.341076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50728 (local address 127.9.236.1:44109)
0929 19:10:29.341460 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:29.341464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:29.341476 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:10:29.341504 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:29.341506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:29.341517 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:29.341868 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:29.342258 (+ 390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:29.342264 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:29.342264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:10:29.342268 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:29.342310 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:10:29.342328 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:29.342425 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:29.342522 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 19:10:29.342540 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":134,"threads_started":1}
I20260929 19:10:29.342667 15232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.340891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46948 (local address 127.9.236.3:41759)
0929 19:10:29.341336 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:29.341343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:29.341353 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:10:29.341376 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:29.341379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:29.341389 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:29.341462 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:29.342391 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:29.342397 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:29.342397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:10:29.342402 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:29.342434 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:10:29.342456 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:29.342518 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:29.342584 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 19:10:29.342598 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":93,"threads_started":1}
I20260929 19:10:29.343169 15231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.340803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:41759 (local address 127.0.0.1:46948)
0929 19:10:29.341046 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:29.341085 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:29.341166 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:29.341771 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:29.341772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:29.341776 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:29.342296 (+ 520us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:10:29.342296 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:29.342308 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:10:29.342317 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:29.342516 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:29.342517 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:29.342518 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 19:10:29.342577 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 19:10:29.342593 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":91,"threads_started":1}
I20260929 19:10:29.343297 15234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.341159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35918 (local address 127.9.236.2:32969)
0929 19:10:29.341374 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:29.341379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:29.341519 (+ 140us) server_negotiation.cc:407] Connection header received
0929 19:10:29.341555 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:29.341558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:29.341569 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:29.341634 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:29.342966 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:29.342974 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:29.342975 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:10:29.342980 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:29.343016 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:10:29.343038 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:29.343106 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:29.343188 (+ 82us) server_negotiation.cc:299] Negotiation successful
0929 19:10:29.343206 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":97,"threads_started":1}
I20260929 19:10:29.343386 14893 tablet_service.cc:1467] Processing CreateTablet for tablet 418888f0b91a4610aa727ffa137b53ee (DEFAULT_TABLE table=test-table [id=6cdc2efa0be8413389bef173cad93a13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:29.343680 14893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 418888f0b91a4610aa727ffa137b53ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:29.343284 15229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.340848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:32969 (local address 127.0.0.1:35918)
0929 19:10:29.341409 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:29.341419 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:29.341458 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:29.341897 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:29.341898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:29.341901 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:29.342857 (+ 956us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:10:29.342857 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:29.342865 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:10:29.342877 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:29.343115 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:29.343116 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:29.343116 (+ 0us) client_negotiation.cc:769] Sending connection context
0929 19:10:29.343168 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 19:10:29.343183 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":131,"threads_started":1}
I20260929 19:10:29.344441 15155 tablet_service.cc:1467] Processing CreateTablet for tablet 418888f0b91a4610aa727ffa137b53ee (DEFAULT_TABLE table=test-table [id=6cdc2efa0be8413389bef173cad93a13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:29.344542 15024 tablet_service.cc:1467] Processing CreateTablet for tablet 418888f0b91a4610aa727ffa137b53ee (DEFAULT_TABLE table=test-table [id=6cdc2efa0be8413389bef173cad93a13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:29.344848 15155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 418888f0b91a4610aa727ffa137b53ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:29.344848 15024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 418888f0b91a4610aa727ffa137b53ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:29.346501 15235 tablet_bootstrap.cc:492] T 418888f0b91a4610aa727ffa137b53ee P 944fc11643044c3995e256b8fc65d464: Bootstrap starting.
I20260929 19:10:29.347100 15235 tablet_bootstrap.cc:654] T 418888f0b91a4610aa727ffa137b53ee P 944fc11643044c3995e256b8fc65d464: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:29.347378 15235 log.cc:824] T 418888f0b91a4610aa727ffa137b53ee P 944fc11643044c3995e256b8fc65d464: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:29.347982 15235 tablet_bootstrap.cc:492] T 418888f0b91a4610aa727ffa137b53ee P 944fc11643044c3995e256b8fc65d464: No bootstrap required, opened a new log
I20260929 19:10:29.348038 15235 ts_tablet_manager.cc:1397] T 418888f0b91a4610aa727ffa137b53ee P 944fc11643044c3995e256b8fc65d464: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:10:29.348417 15237 tablet_bootstrap.cc:492] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d: Bootstrap starting.
I20260929 19:10:29.349155 15237 tablet_bootstrap.cc:654] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:29.349273 15238 tablet_bootstrap.cc:492] T 418888f0b91a4610aa727ffa137b53ee P cc409ffbf92f48a69f51579e576a7881: Bootstrap starting.
I20260929 19:10:29.349551 15237 log.cc:824] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:29.349937 15238 tablet_bootstrap.cc:654] T 418888f0b91a4610aa727ffa137b53ee P cc409ffbf92f48a69f51579e576a7881: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:29.350281 15238 log.cc:824] T 418888f0b91a4610aa727ffa137b53ee P cc409ffbf92f48a69f51579e576a7881: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:29.350399 15237 tablet_bootstrap.cc:492] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d: No bootstrap required, opened a new log
I20260929 19:10:29.350476 15237 ts_tablet_manager.cc:1397] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 19:10:29.350647 15235 raft_consensus.cc:348] T 418888f0b91a4610aa727ffa137b53ee P 944fc11643044c3995e256b8fc65d464 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc409ffbf92f48a69f51579e576a7881" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 41759 } } peers { permanent_uuid: "fd1e995782f84d5db04509225f1a173d" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 32969 } } peers { permanent_uuid: "944fc11643044c3995e256b8fc65d464" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44109 } }
I20260929 19:10:29.350816 15235 raft_consensus.cc:374] T 418888f0b91a4610aa727ffa137b53ee P 944fc11643044c3995e256b8fc65d464 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:29.350853 15235 raft_consensus.cc:729] T 418888f0b91a4610aa727ffa137b53ee P 944fc11643044c3995e256b8fc65d464 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 944fc11643044c3995e256b8fc65d464, State: Initialized, Role: FOLLOWER
I20260929 19:10:29.350977 15235 consensus_queue.cc:260] T 418888f0b91a4610aa727ffa137b53ee P 944fc11643044c3995e256b8fc65d464 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc409ffbf92f48a69f51579e576a7881" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 41759 } } peers { permanent_uuid: "fd1e995782f84d5db04509225f1a173d" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 32969 } } peers { permanent_uuid: "944fc11643044c3995e256b8fc65d464" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44109 } }
I20260929 19:10:29.351236 15238 tablet_bootstrap.cc:492] T 418888f0b91a4610aa727ffa137b53ee P cc409ffbf92f48a69f51579e576a7881: No bootstrap required, opened a new log
I20260929 19:10:29.351235 14959 heartbeater.cc:503] Master 127.9.236.62:46309 was elected leader, sending a full tablet report...
I20260929 19:10:29.351342 15238 ts_tablet_manager.cc:1397] T 418888f0b91a4610aa727ffa137b53ee P cc409ffbf92f48a69f51579e576a7881: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:10:29.351227 15235 ts_tablet_manager.cc:1428] T 418888f0b91a4610aa727ffa137b53ee P 944fc11643044c3995e256b8fc65d464: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:10:29.353470 15237 raft_consensus.cc:348] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc409ffbf92f48a69f51579e576a7881" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 41759 } } peers { permanent_uuid: "fd1e995782f84d5db04509225f1a173d" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 32969 } } peers { permanent_uuid: "944fc11643044c3995e256b8fc65d464" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44109 } }
I20260929 19:10:29.353720 15237 raft_consensus.cc:374] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:29.353761 15237 raft_consensus.cc:729] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd1e995782f84d5db04509225f1a173d, State: Initialized, Role: FOLLOWER
I20260929 19:10:29.353893 15237 consensus_queue.cc:260] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc409ffbf92f48a69f51579e576a7881" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 41759 } } peers { permanent_uuid: "fd1e995782f84d5db04509225f1a173d" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 32969 } } peers { permanent_uuid: "944fc11643044c3995e256b8fc65d464" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44109 } }
I20260929 19:10:29.354173 15237 ts_tablet_manager.cc:1428] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260929 19:10:29.354094 15238 raft_consensus.cc:348] T 418888f0b91a4610aa727ffa137b53ee P cc409ffbf92f48a69f51579e576a7881 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc409ffbf92f48a69f51579e576a7881" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 41759 } } peers { permanent_uuid: "fd1e995782f84d5db04509225f1a173d" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 32969 } } peers { permanent_uuid: "944fc11643044c3995e256b8fc65d464" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44109 } }
I20260929 19:10:29.354198 15090 heartbeater.cc:503] Master 127.9.236.62:46309 was elected leader, sending a full tablet report...
I20260929 19:10:29.354239 15238 raft_consensus.cc:374] T 418888f0b91a4610aa727ffa137b53ee P cc409ffbf92f48a69f51579e576a7881 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:29.354281 15238 raft_consensus.cc:729] T 418888f0b91a4610aa727ffa137b53ee P cc409ffbf92f48a69f51579e576a7881 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc409ffbf92f48a69f51579e576a7881, State: Initialized, Role: FOLLOWER
I20260929 19:10:29.354396 15238 consensus_queue.cc:260] T 418888f0b91a4610aa727ffa137b53ee P cc409ffbf92f48a69f51579e576a7881 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc409ffbf92f48a69f51579e576a7881" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 41759 } } peers { permanent_uuid: "fd1e995782f84d5db04509225f1a173d" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 32969 } } peers { permanent_uuid: "944fc11643044c3995e256b8fc65d464" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44109 } }
I20260929 19:10:29.354658 15238 ts_tablet_manager.cc:1428] T 418888f0b91a4610aa727ffa137b53ee P cc409ffbf92f48a69f51579e576a7881: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:10:29.354714 15221 heartbeater.cc:503] Master 127.9.236.62:46309 was elected leader, sending a full tablet report...
I20260929 19:10:29.411454 15242 raft_consensus.cc:482] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:10:29.411579 15242 raft_consensus.cc:504] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc409ffbf92f48a69f51579e576a7881" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 41759 } } peers { permanent_uuid: "fd1e995782f84d5db04509225f1a173d" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 32969 } } peers { permanent_uuid: "944fc11643044c3995e256b8fc65d464" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44109 } }
I20260929 19:10:29.411909 15242 leader_election.cc:288] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc409ffbf92f48a69f51579e576a7881 (127.9.236.3:41759), 944fc11643044c3995e256b8fc65d464 (127.9.236.1:44109)
I20260929 19:10:29.413417 15092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.412151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:44109 (local address 127.9.236.2:46741)
0929 19:10:29.412215 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:29.412228 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:29.412272 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:29.412490 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:29.412492 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:29.412497 (+ 5us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:29.413043 (+ 546us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:10:29.413044 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:29.413052 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:10:29.413065 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:29.413295 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:29.413296 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:29.413297 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 19:10:29.413336 (+ 39us) client_negotiation.cc:240] Negotiation successful
0929 19:10:29.413351 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260929 19:10:29.413595 15244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.412151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:41759 (local address 127.9.236.2:46447)
0929 19:10:29.412502 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:29.412511 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:29.412564 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:29.412700 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:29.412701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:29.412719 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:29.413206 (+ 487us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:10:29.413207 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:29.413215 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:10:29.413231 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:29.413453 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:29.413454 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:29.413454 (+ 0us) client_negotiation.cc:769] Sending connection context
0929 19:10:29.413514 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 19:10:29.413528 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":178,"thread_start_us":104,"threads_started":1}
I20260929 19:10:29.413595 15233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.412200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:46741 (local address 127.9.236.1:44109)
0929 19:10:29.412259 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:29.412261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:29.412273 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:10:29.412405 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:29.412407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:29.412413 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:29.412466 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:29.413204 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:29.413205 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:29.413206 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:10:29.413210 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:29.413226 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:10:29.413231 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:29.413274 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:29.413522 (+ 248us) server_negotiation.cc:299] Negotiation successful
0929 19:10:29.413535 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 19:10:29.413718 15232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.412303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:46447 (local address 127.9.236.3:41759)
0929 19:10:29.412374 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:29.412377 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:29.412583 (+ 206us) server_negotiation.cc:407] Connection header received
0929 19:10:29.412612 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:29.412614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:29.412619 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:29.412681 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:29.413315 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:29.413316 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:29.413317 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:10:29.413320 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:29.413364 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:10:29.413369 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:29.413424 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:29.413607 (+ 183us) server_negotiation.cc:299] Negotiation successful
0929 19:10:29.413627 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 19:10:29.413882 14913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "418888f0b91a4610aa727ffa137b53ee" candidate_uuid: "fd1e995782f84d5db04509225f1a173d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "944fc11643044c3995e256b8fc65d464" is_pre_election: true
I20260929 19:10:29.414031 14913 raft_consensus.cc:2454] T 418888f0b91a4610aa727ffa137b53ee P 944fc11643044c3995e256b8fc65d464 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd1e995782f84d5db04509225f1a173d in term 0.
I20260929 19:10:29.414100 15175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "418888f0b91a4610aa727ffa137b53ee" candidate_uuid: "fd1e995782f84d5db04509225f1a173d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc409ffbf92f48a69f51579e576a7881" is_pre_election: true
I20260929 19:10:29.414270 14979 leader_election.cc:302] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 944fc11643044c3995e256b8fc65d464, fd1e995782f84d5db04509225f1a173d; no voters:
I20260929 19:10:29.414302 15175 raft_consensus.cc:2454] T 418888f0b91a4610aa727ffa137b53ee P cc409ffbf92f48a69f51579e576a7881 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd1e995782f84d5db04509225f1a173d in term 0.
I20260929 19:10:29.414434 15242 raft_consensus.cc:2781] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:10:29.414494 15242 raft_consensus.cc:482] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:10:29.414510 15242 raft_consensus.cc:3037] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:29.415360 15242 raft_consensus.cc:504] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc409ffbf92f48a69f51579e576a7881" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 41759 } } peers { permanent_uuid: "fd1e995782f84d5db04509225f1a173d" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 32969 } } peers { permanent_uuid: "944fc11643044c3995e256b8fc65d464" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44109 } }
I20260929 19:10:29.415639 15242 leader_election.cc:288] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d [CANDIDATE]: Term 1 election: Requested vote from peers cc409ffbf92f48a69f51579e576a7881 (127.9.236.3:41759), 944fc11643044c3995e256b8fc65d464 (127.9.236.1:44109)
I20260929 19:10:29.415671 15175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "418888f0b91a4610aa727ffa137b53ee" candidate_uuid: "fd1e995782f84d5db04509225f1a173d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc409ffbf92f48a69f51579e576a7881"
I20260929 19:10:29.415741 15175 raft_consensus.cc:3037] T 418888f0b91a4610aa727ffa137b53ee P cc409ffbf92f48a69f51579e576a7881 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:29.415707 14913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "418888f0b91a4610aa727ffa137b53ee" candidate_uuid: "fd1e995782f84d5db04509225f1a173d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "944fc11643044c3995e256b8fc65d464"
I20260929 19:10:29.415882 14913 raft_consensus.cc:3037] T 418888f0b91a4610aa727ffa137b53ee P 944fc11643044c3995e256b8fc65d464 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:29.416586 15175 raft_consensus.cc:2454] T 418888f0b91a4610aa727ffa137b53ee P cc409ffbf92f48a69f51579e576a7881 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd1e995782f84d5db04509225f1a173d in term 1.
I20260929 19:10:29.416700 14913 raft_consensus.cc:2454] T 418888f0b91a4610aa727ffa137b53ee P 944fc11643044c3995e256b8fc65d464 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd1e995782f84d5db04509225f1a173d in term 1.
I20260929 19:10:29.416800 14980 leader_election.cc:302] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc409ffbf92f48a69f51579e576a7881, fd1e995782f84d5db04509225f1a173d; no voters:
I20260929 19:10:29.416896 15242 raft_consensus.cc:2781] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:29.417044 15242 raft_consensus.cc:686] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d [term 1 LEADER]: Becoming Leader. State: Replica: fd1e995782f84d5db04509225f1a173d, State: Running, Role: LEADER
I20260929 19:10:29.417174 15242 consensus_queue.cc:237] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc409ffbf92f48a69f51579e576a7881" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 41759 } } peers { permanent_uuid: "fd1e995782f84d5db04509225f1a173d" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 32969 } } peers { permanent_uuid: "944fc11643044c3995e256b8fc65d464" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44109 } }
I20260929 19:10:29.418098 14770 catalog_manager.cc:5495] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d reported cstate change: term changed from 0 to 1, leader changed from <none> to fd1e995782f84d5db04509225f1a173d (127.9.236.2). New cstate: current_term: 1 leader_uuid: "fd1e995782f84d5db04509225f1a173d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc409ffbf92f48a69f51579e576a7881" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 41759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd1e995782f84d5db04509225f1a173d" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 32969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "944fc11643044c3995e256b8fc65d464" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44109 } health_report { overall_health: UNKNOWN } } }
I20260929 19:10:29.435052 15234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.434093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35920 (local address 127.9.236.2:32969)
0929 19:10:29.434168 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:29.434170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:29.434178 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:10:29.434220 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:29.434221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:29.434235 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:29.434281 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:29.434835 (+ 554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:29.434837 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:29.434837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:10:29.434841 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:29.434857 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:10:29.434861 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:29.434911 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:29.434954 (+ 43us) server_negotiation.cc:299] Negotiation successful
0929 19:10:29.434969 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 19:10:29.436944 14913 raft_consensus.cc:1261] T 418888f0b91a4610aa727ffa137b53ee P 944fc11643044c3995e256b8fc65d464 [term 1 FOLLOWER]: Refusing update from remote peer fd1e995782f84d5db04509225f1a173d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:29.437008 15175 raft_consensus.cc:1261] T 418888f0b91a4610aa727ffa137b53ee P cc409ffbf92f48a69f51579e576a7881 [term 1 FOLLOWER]: Refusing update from remote peer fd1e995782f84d5db04509225f1a173d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:29.437340 15245 consensus_queue.cc:1035] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d [LEADER]: Connected to new peer: Peer: permanent_uuid: "944fc11643044c3995e256b8fc65d464" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:29.437486 15242 consensus_queue.cc:1035] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc409ffbf92f48a69f51579e576a7881" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 41759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:29.443105 14769 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:29.443193 14769 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:29.444033 14769 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9b4caf61ff6249a187840d6da32ef210: Sending DeleteTablet for 3 replicas of tablet 418888f0b91a4610aa727ffa137b53ee
I20260929 19:10:29.444366 15155 tablet_service.cc:1514] Processing DeleteTablet for tablet 418888f0b91a4610aa727ffa137b53ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:29 UTC) from {username='slave'} at 127.0.0.1:46948
I20260929 19:10:29.444458 14893 tablet_service.cc:1514] Processing DeleteTablet for tablet 418888f0b91a4610aa727ffa137b53ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:29 UTC) from {username='slave'} at 127.0.0.1:50728
I20260929 19:10:29.444687 15259 tablet_replica.cc:330] T 418888f0b91a4610aa727ffa137b53ee P 944fc11643044c3995e256b8fc65d464: stopping tablet replica
I20260929 19:10:29.444873 15259 raft_consensus.cc:2229] T 418888f0b91a4610aa727ffa137b53ee P 944fc11643044c3995e256b8fc65d464 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:29.444861 15024 tablet_service.cc:1514] Processing DeleteTablet for tablet 418888f0b91a4610aa727ffa137b53ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:29 UTC) from {username='slave'} at 127.0.0.1:35918
I20260929 19:10:29.444929 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 14831
I20260929 19:10:29.445050 15260 tablet_replica.cc:330] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d: stopping tablet replica
I20260929 19:10:29.445192 15259 raft_consensus.cc:2258] T 418888f0b91a4610aa727ffa137b53ee P 944fc11643044c3995e256b8fc65d464 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:29.445704 15260 raft_consensus.cc:2229] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:10:29.445863 15260 raft_consensus.cc:2258] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:29.446321 15260 ts_tablet_manager.cc:1905] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:29.448097 15260 ts_tablet_manager.cc:1918] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:10:29.448165 15260 log.cc:1196] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/418888f0b91a4610aa727ffa137b53ee
I20260929 19:10:29.448415 15260 ts_tablet_manager.cc:1939] T 418888f0b91a4610aa727ffa137b53ee P fd1e995782f84d5db04509225f1a173d: Deleting consensus metadata
I20260929 19:10:29.448773 14755 catalog_manager.cc:4841] TS fd1e995782f84d5db04509225f1a173d (127.9.236.2:32969): tablet 418888f0b91a4610aa727ffa137b53ee (table test-table [id=6cdc2efa0be8413389bef173cad93a13]) successfully deleted
I20260929 19:10:29.449275 15258 tablet_replica.cc:330] T 418888f0b91a4610aa727ffa137b53ee P cc409ffbf92f48a69f51579e576a7881: stopping tablet replica
I20260929 19:10:29.449399 15258 raft_consensus.cc:2229] T 418888f0b91a4610aa727ffa137b53ee P cc409ffbf92f48a69f51579e576a7881 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:29.449507 15258 raft_consensus.cc:2258] T 418888f0b91a4610aa727ffa137b53ee P cc409ffbf92f48a69f51579e576a7881 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:29.450466 15258 ts_tablet_manager.cc:1905] T 418888f0b91a4610aa727ffa137b53ee P cc409ffbf92f48a69f51579e576a7881: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 19:10:29.451335 14756 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.236.1:44109 (error 108)
W20260929 19:10:29.451747 15229 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:29.451642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:44109 (local address 127.0.0.1:50738)
0929 19:10:29.451684 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:29.451714 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.1:44109: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":25}
I20260929 19:10:29.451892 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 14962
W20260929 19:10:29.451970 14756 catalog_manager.cc:4584] TS 944fc11643044c3995e256b8fc65d464 (127.9.236.1:44109): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 418888f0b91a4610aa727ffa137b53ee: Network error: Client connection negotiation failed: client connection to 127.9.236.1:44109: connect: Connection reset by peer (error 104)
I20260929 19:10:29.452772 15258 ts_tablet_manager.cc:1918] T 418888f0b91a4610aa727ffa137b53ee P cc409ffbf92f48a69f51579e576a7881: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:10:29.452850 15258 log.cc:1196] T 418888f0b91a4610aa727ffa137b53ee P cc409ffbf92f48a69f51579e576a7881: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/418888f0b91a4610aa727ffa137b53ee
I20260929 19:10:29.453078 15258 ts_tablet_manager.cc:1939] T 418888f0b91a4610aa727ffa137b53ee P cc409ffbf92f48a69f51579e576a7881: Deleting consensus metadata
I20260929 19:10:29.453507 14757 catalog_manager.cc:4841] TS cc409ffbf92f48a69f51579e576a7881 (127.9.236.3:41759): tablet 418888f0b91a4610aa727ffa137b53ee (table test-table [id=6cdc2efa0be8413389bef173cad93a13]) successfully deleted
I20260929 19:10:29.456247 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 15093
I20260929 19:10:29.460798 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 14739
2026-09-29T19:10:29Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (694 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260929 19:10:29.479437 10160 mini_oidc.cc:152] Starting JWKS server
I20260929 19:10:29.480420 10160 webserver.cc:460] Webserver started at https://127.0.0.1:38315/ using document root <none> and password file <none>
I20260929 19:10:29.480513 10160 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 19:10:29.480657 10160 webserver.cc:460] Webserver started at http://127.0.0.1:38459/ using document root <none> and password file <none>
2026-09-29T19:10:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:10:29Z Disabled control of system clock
I20260929 19:10:29.488412 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:35383
--webserver_interface=127.9.236.62
--webserver_port=0
--builtin_ntp_servers=127.9.236.20:43815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:35383
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38315/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790709008521303-10160-0/testchainca.cert with env {}
W20260929 19:10:29.565376 15267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:29.565586 15267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:29.565608 15267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:29.566720 15267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:29.566758 15267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:29.566771 15267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:29.566781 15267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:29.566792 15267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 19:10:29.566803 15267 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 19:10:29.566814 15267 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38315/jwks.json
I20260929 19:10:29.568362 15267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:43815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:35383
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:35383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38315/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790709008521303-10160-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:29.568552 15267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:29.568707 15267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:29.573817 15275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:29.573935 15273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:29.573798 15272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:29.574221 15267 server_base.cc:1034] running on GCE node
I20260929 19:10:29.574482 15267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:29.574806 15267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:29.575954 15267 hybrid_clock.cc:648] HybridClock initialized: now 1790709029575929 us; error 43 us; skew 500 ppm
I20260929 19:10:29.577322 15267 webserver.cc:460] Webserver started at http://127.9.236.62:45523/ using document root <none> and password file <none>
I20260929 19:10:29.577564 15267 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:29.577616 15267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:29.577755 15267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:29.578912 15267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "1751643f6c2f41aabd536bc87ccd62ef"
format_stamp: "Formatted at 2026-09-29 19:10:29 on dist-test-slave-wftq"
server_key: "42e07da886fae16381cbb03e6eb473f9"
server_key_iv: "f16fb4aece4712f122f880377fe5558a"
server_key_version: "encryptionkey@0"
I20260929 19:10:29.579268 15267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "1751643f6c2f41aabd536bc87ccd62ef"
format_stamp: "Formatted at 2026-09-29 19:10:29 on dist-test-slave-wftq"
server_key: "42e07da886fae16381cbb03e6eb473f9"
server_key_iv: "f16fb4aece4712f122f880377fe5558a"
server_key_version: "encryptionkey@0"
I20260929 19:10:29.580611 15267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:10:29.581425 15281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:29.581604 15267 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:29.581700 15267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "1751643f6c2f41aabd536bc87ccd62ef"
format_stamp: "Formatted at 2026-09-29 19:10:29 on dist-test-slave-wftq"
server_key: "42e07da886fae16381cbb03e6eb473f9"
server_key_iv: "f16fb4aece4712f122f880377fe5558a"
server_key_version: "encryptionkey@0"
I20260929 19:10:29.581774 15267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:29.595803 15267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:29.596179 15267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:29.596329 15267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:29.601174 15267 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:35383
I20260929 19:10:29.601244 15333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:35383 every 8 connection(s)
I20260929 19:10:29.601588 15267 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:10:29.602595 15334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:29.602972 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 15267
I20260929 19:10:29.603089 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:10:29.603271 10160 external_mini_cluster.cc:1428] Setting key 68ca5782acd0cb49abe19a14449e59d3
I20260929 19:10:29.605480 15334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef: Bootstrap starting.
I20260929 19:10:29.606048 15334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:29.606307 15334 log.cc:824] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:29.606906 15334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef: No bootstrap required, opened a new log
I20260929 19:10:29.609254 15334 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1751643f6c2f41aabd536bc87ccd62ef" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35383 } }
I20260929 19:10:29.609418 15334 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:29.609464 15334 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1751643f6c2f41aabd536bc87ccd62ef, State: Initialized, Role: FOLLOWER
I20260929 19:10:29.609515 15337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.603865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43254 (local address 127.9.236.62:35383)
0929 19:10:29.604119 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:29.604128 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:29.604156 (+ 28us) server_negotiation.cc:407] Connection header received
0929 19:10:29.604702 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:29.604710 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:29.604777 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:29.604920 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:29.605206 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:29.606210 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:29.607125 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:29.607777 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:29.607922 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:29.608755 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:29.608760 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:29.608761 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:10:29.608984 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:29.609026 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:10:29.609040 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:29.609141 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:29.609221 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 19:10:29.609304 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":100,"threads_started":1}
I20260929 19:10:29.609768 15334 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1751643f6c2f41aabd536bc87ccd62ef" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35383 } }
I20260929 19:10:29.609848 15334 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:10:29.609870 15334 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:10:29.609920 15334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:29.610423 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:10:29.610489 15334 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1751643f6c2f41aabd536bc87ccd62ef" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35383 } }
I20260929 19:10:29.610572 15334 leader_election.cc:302] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1751643f6c2f41aabd536bc87ccd62ef; no voters:
I20260929 19:10:29.610718 15334 leader_election.cc:288] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:10:29.610796 15339 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:29.610981 15339 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef [term 1 LEADER]: Becoming Leader. State: Replica: 1751643f6c2f41aabd536bc87ccd62ef, State: Running, Role: LEADER
I20260929 19:10:29.611016 15334 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:29.611099 15339 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1751643f6c2f41aabd536bc87ccd62ef" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35383 } }
I20260929 19:10:29.611712 15340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1751643f6c2f41aabd536bc87ccd62ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1751643f6c2f41aabd536bc87ccd62ef" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35383 } } }
I20260929 19:10:29.611726 15341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1751643f6c2f41aabd536bc87ccd62ef. Latest consensus state: current_term: 1 leader_uuid: "1751643f6c2f41aabd536bc87ccd62ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1751643f6c2f41aabd536bc87ccd62ef" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35383 } } }
I20260929 19:10:29.611969 15341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:29.611934 15340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:29.612468 15350 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:29.613003 15350 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:10:29.614660 15350 catalog_manager.cc:1347] Generated new cluster ID: 0c735992ed9a4f0d85113c375c0544cc
I20260929 19:10:29.614717 15350 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:10:29.628768 15350 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:10:29.629379 15350 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:29.636868 15350 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1751643f6c2f41aabd536bc87ccd62ef: Generated new TSK 0
I20260929 19:10:29.637147 15350 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 19:10:29.666517 15364 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260929 19:10:29.666611 15364 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 19:10:29.666963 15337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.650764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43268 (local address 127.9.236.62:35383)
0929 19:10:29.650818 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:29.650820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:29.650923 (+ 103us) server_negotiation.cc:407] Connection header received
0929 19:10:29.651011 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:29.651012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:29.651026 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:29.651121 (+ 95us) server_negotiation.cc:226] Negotiated authn=JWT
0929 19:10:29.651255 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:29.652063 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:29.652860 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:29.653204 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:29.653288 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:29.653486 (+ 198us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 19:10:29.666678 (+ 13192us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38315/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 19:10:29.666872 (+ 194us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43268: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38315/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":30}
W20260929 19:10:29.667083 15337 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43268: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38315/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260929 19:10:29.667246 15363 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:29.650670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:35383 (local address 127.0.0.1:43268)
0929 19:10:29.650842 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:29.650852 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:29.650917 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:29.651123 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:29.651124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:29.651126 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0929 19:10:29.651190 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:29.651192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:29.652157 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:29.652158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:29.652776 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:29.652777 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:29.653282 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:29.653283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:29.653348 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:29.653366 (+ 18us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 19:10:29.666834 (+ 13468us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38315/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 19:10:29.666948 (+ 114us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.236.62:35383: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38315/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.9.236.62:35383: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38315/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260929 19:10:29.667737 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 15267
2026-09-29T19:10:29Z chronyd exiting
I20260929 19:10:29.704074 10160 test_util.cc:175] -----------------------------------------------
I20260929 19:10:29.704159 10160 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790709008521303-10160-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (225 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260929 19:10:29.704993 10160 mini_oidc.cc:152] Starting JWKS server
I20260929 19:10:29.705729 10160 webserver.cc:460] Webserver started at https://127.0.0.1:34349/ using document root <none> and password file <none>
I20260929 19:10:29.705775 10160 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 19:10:29.705914 10160 webserver.cc:460] Webserver started at http://127.0.0.1:40703/ using document root <none> and password file <none>
2026-09-29T19:10:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:10:29Z Disabled control of system clock
I20260929 19:10:29.713951 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:35289
--webserver_interface=127.9.236.62
--webserver_port=0
--builtin_ntp_servers=127.9.236.20:35491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:35289
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34349/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790709008521303-10160-0/testchainca.cert with env {}
W20260929 19:10:29.791142 15370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:29.791527 15370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:29.791572 15370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:29.792806 15370 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:29.792848 15370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:29.792860 15370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:29.792871 15370 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:29.792882 15370 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 19:10:29.792893 15370 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 19:10:29.792912 15370 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34349/jwks.json
I20260929 19:10:29.794435 15370 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:35491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:35289
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:35289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34349/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790709008521303-10160-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:29.794625 15370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:29.794785 15370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:29.799391 15376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:29.799396 15375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:29.799513 15378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:29.799657 15370 server_base.cc:1034] running on GCE node
I20260929 19:10:29.799871 15370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:29.800127 15370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:29.801266 15370 hybrid_clock.cc:648] HybridClock initialized: now 1790709029801244 us; error 45 us; skew 500 ppm
I20260929 19:10:29.802552 15370 webserver.cc:460] Webserver started at http://127.9.236.62:43315/ using document root <none> and password file <none>
I20260929 19:10:29.802788 15370 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:29.802845 15370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:29.802963 15370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:29.803975 15370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "36435d5dbf374b3f830429084f2c6014"
format_stamp: "Formatted at 2026-09-29 19:10:29 on dist-test-slave-wftq"
server_key: "cd46da1ca9c39f83b3cbf9528e5e122f"
server_key_iv: "c2d59a7961162d0209714bb7dad895e3"
server_key_version: "encryptionkey@0"
I20260929 19:10:29.804288 15370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "36435d5dbf374b3f830429084f2c6014"
format_stamp: "Formatted at 2026-09-29 19:10:29 on dist-test-slave-wftq"
server_key: "cd46da1ca9c39f83b3cbf9528e5e122f"
server_key_iv: "c2d59a7961162d0209714bb7dad895e3"
server_key_version: "encryptionkey@0"
I20260929 19:10:29.805624 15370 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:10:29.806363 15384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:29.806572 15370 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:29.806684 15370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "36435d5dbf374b3f830429084f2c6014"
format_stamp: "Formatted at 2026-09-29 19:10:29 on dist-test-slave-wftq"
server_key: "cd46da1ca9c39f83b3cbf9528e5e122f"
server_key_iv: "c2d59a7961162d0209714bb7dad895e3"
server_key_version: "encryptionkey@0"
I20260929 19:10:29.806775 15370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:29.823117 15370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:29.823432 15370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:29.823565 15370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:29.828033 15370 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:35289
I20260929 19:10:29.828115 15436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:35289 every 8 connection(s)
I20260929 19:10:29.828462 15370 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:10:29.829218 15437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:29.831851 15437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014: Bootstrap starting.
I20260929 19:10:29.832451 15437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:29.832746 15437 log.cc:824] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:29.833690 15437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014: No bootstrap required, opened a new log
I20260929 19:10:29.835155 15437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36435d5dbf374b3f830429084f2c6014" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35289 } }
I20260929 19:10:29.835289 15437 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:29.835309 15437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36435d5dbf374b3f830429084f2c6014, State: Initialized, Role: FOLLOWER
I20260929 19:10:29.835433 15437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36435d5dbf374b3f830429084f2c6014" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35289 } }
I20260929 19:10:29.835484 15437 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:10:29.835502 15437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:10:29.835534 15437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:29.835987 15437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36435d5dbf374b3f830429084f2c6014" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35289 } }
I20260929 19:10:29.836068 15437 leader_election.cc:302] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36435d5dbf374b3f830429084f2c6014; no voters:
I20260929 19:10:29.836200 15437 leader_election.cc:288] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:10:29.836266 15440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:29.836439 15437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:29.836440 15440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014 [term 1 LEADER]: Becoming Leader. State: Replica: 36435d5dbf374b3f830429084f2c6014, State: Running, Role: LEADER
I20260929 19:10:29.836649 15440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36435d5dbf374b3f830429084f2c6014" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35289 } }
I20260929 19:10:29.837077 15442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36435d5dbf374b3f830429084f2c6014. Latest consensus state: current_term: 1 leader_uuid: "36435d5dbf374b3f830429084f2c6014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36435d5dbf374b3f830429084f2c6014" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35289 } } }
I20260929 19:10:29.837211 15442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:29.837311 15441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36435d5dbf374b3f830429084f2c6014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36435d5dbf374b3f830429084f2c6014" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35289 } } }
I20260929 19:10:29.837400 15441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:29.837525 15449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:29.838039 15449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:10:29.838320 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 15370
I20260929 19:10:29.838420 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:10:29.838603 10160 external_mini_cluster.cc:1428] Setting key e76cf03683e9b5a999e1d378a4743805
I20260929 19:10:29.839687 15449 catalog_manager.cc:1347] Generated new cluster ID: c8cf9ccabb60461dace17fb5ae10e95e
I20260929 19:10:29.839744 15449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:10:29.845206 15459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.839058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60472 (local address 127.9.236.62:35289)
0929 19:10:29.839277 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:29.839287 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:29.839468 (+ 181us) server_negotiation.cc:407] Connection header received
0929 19:10:29.839592 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:29.839599 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:29.839675 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:29.839806 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:29.840125 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:29.841625 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:29.842709 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:29.843366 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:29.843498 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:29.844262 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:29.844266 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:29.844267 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:10:29.844505 (+ 238us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:29.844552 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:10:29.844569 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:29.844641 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:29.844713 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 19:10:29.844801 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":92,"threads_started":1}
I20260929 19:10:29.846527 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:10:29.867239 15449 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:10:29.867771 15449 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:29.870479 15449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36435d5dbf374b3f830429084f2c6014: Generated new TSK 0
I20260929 19:10:29.870615 15449 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 19:10:29.892984 15467 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260929 19:10:29.893067 15467 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 19:10:29.893422 15459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:29.879821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60480 (local address 127.9.236.62:35289)
0929 19:10:29.879880 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:29.879882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:29.879891 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:29.879955 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:29.879956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:29.879970 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:29.880033 (+ 63us) server_negotiation.cc:226] Negotiated authn=JWT
0929 19:10:29.880164 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:29.880650 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:29.881441 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:29.881650 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:29.881702 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:29.881860 (+ 158us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 19:10:29.893204 (+ 11344us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34349/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 19:10:29.893354 (+ 150us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60480: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34349/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":35}
W20260929 19:10:29.893486 15466 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:29.879706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:35289 (local address 127.0.0.1:60480)
0929 19:10:29.879832 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:29.879842 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:29.879894 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:29.880035 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:29.880036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:29.880041 (+ 5us) client_negotiation.cc:189] Negotiated authn=JWT
0929 19:10:29.880105 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:29.880108 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:29.880730 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:29.880731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:29.881371 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:29.881373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:29.881705 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:29.881706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:29.881771 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:29.881776 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 19:10:29.893306 (+ 11530us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34349/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 19:10:29.893398 (+ 92us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.236.62:35289: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34349/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
W20260929 19:10:29.893484 15459 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60480: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34349/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260929 19:10:29.893762 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 15370
2026-09-29T19:10:29Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (225 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260929 19:10:29.930433 10160 mini_oidc.cc:152] Starting JWKS server
I20260929 19:10:29.931082 10160 webserver.cc:460] Webserver started at https://127.0.0.1:44429/ using document root <none> and password file <none>
I20260929 19:10:29.931123 10160 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 19:10:29.931247 10160 webserver.cc:460] Webserver started at http://127.0.0.1:42255/ using document root <none> and password file <none>
2026-09-29T19:10:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:10:29Z Disabled control of system clock
I20260929 19:10:29.939005 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:42855
--webserver_interface=127.9.236.62
--webserver_port=0
--builtin_ntp_servers=127.9.236.20:42707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:42855
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44429/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 19:10:30.033775 15473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:30.033998 15473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:30.034019 15473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:30.035130 15473 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:30.035168 15473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:30.035179 15473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:30.035190 15473 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:30.035200 15473 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 19:10:30.035211 15473 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 19:10:30.035221 15473 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44429/jwks.json
I20260929 19:10:30.036475 15473 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:42707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:42855
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:42855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44429/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:30.036664 15473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:30.036875 15473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:30.041986 15479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:30.042011 15481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:30.042107 15478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:30.042165 15473 server_base.cc:1034] running on GCE node
I20260929 19:10:30.042376 15473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:30.042627 15473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:30.043766 15473 hybrid_clock.cc:648] HybridClock initialized: now 1790709030043747 us; error 48 us; skew 500 ppm
I20260929 19:10:30.049762 15473 webserver.cc:460] Webserver started at http://127.9.236.62:46595/ using document root <none> and password file <none>
I20260929 19:10:30.049980 15473 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:30.050027 15473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:30.050120 15473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:30.051151 15473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "bcb48725e92549eabdcc2f53eb9eba80"
format_stamp: "Formatted at 2026-09-29 19:10:30 on dist-test-slave-wftq"
server_key: "bc6328a50e2f38fea7ff6a2ac3bbc610"
server_key_iv: "3e08ef10f8a7783de7c1672c94f745e5"
server_key_version: "encryptionkey@0"
I20260929 19:10:30.051434 15473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "bcb48725e92549eabdcc2f53eb9eba80"
format_stamp: "Formatted at 2026-09-29 19:10:30 on dist-test-slave-wftq"
server_key: "bc6328a50e2f38fea7ff6a2ac3bbc610"
server_key_iv: "3e08ef10f8a7783de7c1672c94f745e5"
server_key_version: "encryptionkey@0"
I20260929 19:10:30.052645 15473 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:10:30.053388 15487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:30.053544 15473 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:30.053627 15473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "bcb48725e92549eabdcc2f53eb9eba80"
format_stamp: "Formatted at 2026-09-29 19:10:30 on dist-test-slave-wftq"
server_key: "bc6328a50e2f38fea7ff6a2ac3bbc610"
server_key_iv: "3e08ef10f8a7783de7c1672c94f745e5"
server_key_version: "encryptionkey@0"
I20260929 19:10:30.053683 15473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:30.062457 15473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:30.062703 15473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:30.062803 15473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:30.066848 15473 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:42855
I20260929 19:10:30.066979 15539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:42855 every 8 connection(s)
I20260929 19:10:30.067239 15473 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:10:30.067996 15540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:30.071175 15540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80: Bootstrap starting.
I20260929 19:10:30.071805 15540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:30.072115 15540 log.cc:824] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:30.072917 15540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80: No bootstrap required, opened a new log
I20260929 19:10:30.073688 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 15473
I20260929 19:10:30.073781 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:10:30.073937 10160 external_mini_cluster.cc:1428] Setting key 9649028f240512d48dd54000e991ec3a
I20260929 19:10:30.074951 15540 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcb48725e92549eabdcc2f53eb9eba80" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 42855 } }
I20260929 19:10:30.075117 15540 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:30.075152 15540 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb48725e92549eabdcc2f53eb9eba80, State: Initialized, Role: FOLLOWER
I20260929 19:10:30.075318 15540 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcb48725e92549eabdcc2f53eb9eba80" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 42855 } }
I20260929 19:10:30.075405 15540 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:10:30.075438 15540 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:10:30.075488 15540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:30.076035 15540 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcb48725e92549eabdcc2f53eb9eba80" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 42855 } }
I20260929 19:10:30.076133 15540 leader_election.cc:302] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcb48725e92549eabdcc2f53eb9eba80; no voters:
I20260929 19:10:30.076295 15540 leader_election.cc:288] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:10:30.076344 15545 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:30.076469 15545 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80 [term 1 LEADER]: Becoming Leader. State: Replica: bcb48725e92549eabdcc2f53eb9eba80, State: Running, Role: LEADER
I20260929 19:10:30.076565 15545 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcb48725e92549eabdcc2f53eb9eba80" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 42855 } }
I20260929 19:10:30.076592 15540 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:30.076962 15545 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcb48725e92549eabdcc2f53eb9eba80. Latest consensus state: current_term: 1 leader_uuid: "bcb48725e92549eabdcc2f53eb9eba80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcb48725e92549eabdcc2f53eb9eba80" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 42855 } } }
I20260929 19:10:30.077030 15545 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:30.077004 15547 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcb48725e92549eabdcc2f53eb9eba80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcb48725e92549eabdcc2f53eb9eba80" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 42855 } } }
I20260929 19:10:30.077183 15547 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:30.077346 15550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:30.077935 15550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:10:30.080005 15550 catalog_manager.cc:1347] Generated new cluster ID: 26fffda8c2994fb28f77c71bdbb2eed4
I20260929 19:10:30.080085 15550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:10:30.080209 15544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:30.074433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45756 (local address 127.9.236.62:42855)
0929 19:10:30.074671 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:30.074679 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:30.074708 (+ 29us) server_negotiation.cc:407] Connection header received
0929 19:10:30.074926 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:30.074932 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:30.075019 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:30.075155 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:30.075414 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:30.076890 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:30.077958 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:30.078544 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:30.078657 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:30.079344 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:30.079349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:30.079350 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:10:30.079591 (+ 241us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:30.079636 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:10:30.079652 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:30.079718 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:30.079836 (+ 118us) server_negotiation.cc:299] Negotiation successful
0929 19:10:30.079928 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":92,"threads_started":1}
I20260929 19:10:30.081157 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:10:30.087909 15550 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:10:30.088701 15550 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:30.093833 15550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bcb48725e92549eabdcc2f53eb9eba80: Generated new TSK 0
I20260929 19:10:30.094038 15550 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:10:30.104854 15544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:30.100741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45772 (local address 127.9.236.62:42855)
0929 19:10:30.100811 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:30.100813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:30.100866 (+ 53us) server_negotiation.cc:407] Connection header received
0929 19:10:30.100943 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:30.100943 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:30.100958 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:30.101056 (+ 98us) server_negotiation.cc:226] Negotiated authn=JWT
0929 19:10:30.101219 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:30.102064 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:30.102879 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:30.103196 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:30.103270 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:30.103470 (+ 200us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 19:10:30.104615 (+ 1145us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44429/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 19:10:30.104760 (+ 145us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45772: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44429/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":39}
W20260929 19:10:30.104930 15544 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45772: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44429/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260929 19:10:30.106844 15569 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:30.100630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:42855 (local address 127.0.0.1:45772)
0929 19:10:30.100811 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:30.100819 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:30.100879 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:30.101067 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:30.101067 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:30.101069 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0929 19:10:30.101153 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:30.101156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:30.102188 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:30.102188 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:30.102792 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:30.102793 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:30.103317 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:30.103317 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:30.103359 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:30.103363 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 19:10:30.104725 (+ 1362us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44429/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 19:10:30.106725 (+ 2000us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.236.62:42855: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44429/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":1813,"thread_start_us":76,"threads_started":1}
I20260929 19:10:30.107146 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 15473
E20260929 19:10:30.115834 15570 webserver.cc:533] Webserver: sslize failed:
E20260929 19:10:30.115899 15570 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-29T19:10:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (213 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-29T19:10:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:10:30Z Disabled control of system clock
I20260929 19:10:30.152351 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:46297
--webserver_interface=127.9.236.62
--webserver_port=0
--builtin_ntp_servers=127.9.236.20:34849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:46297
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790709008521303-10160-0/testchainca.cert with env {}
W20260929 19:10:30.239820 15574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:30.240060 15574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:30.240092 15574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:30.241442 15574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:30.241487 15574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:30.241503 15574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:30.241519 15574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:30.241535 15574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 19:10:30.241568 15574 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 19:10:30.241585 15574 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260929 19:10:30.243187 15574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:34849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:46297
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:46297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790709008521303-10160-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:30.243455 15574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:30.243672 15574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:30.248972 15580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:30.248982 15579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:30.249024 15582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:30.249214 15574 server_base.cc:1034] running on GCE node
I20260929 19:10:30.249567 15574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:30.249876 15574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:30.251026 15574 hybrid_clock.cc:648] HybridClock initialized: now 1790709030251004 us; error 47 us; skew 500 ppm
I20260929 19:10:30.252316 15574 webserver.cc:460] Webserver started at http://127.9.236.62:38835/ using document root <none> and password file <none>
I20260929 19:10:30.252534 15574 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:30.252581 15574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:30.252708 15574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:30.253895 15574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "47a5c2e0b45d49cba55a3b081516c558"
format_stamp: "Formatted at 2026-09-29 19:10:30 on dist-test-slave-wftq"
server_key: "ed17c9a03570527200eadfe8b1ca7d03"
server_key_iv: "aba21b7194d9524f771c395c8bdb97bd"
server_key_version: "encryptionkey@0"
I20260929 19:10:30.254245 15574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "47a5c2e0b45d49cba55a3b081516c558"
format_stamp: "Formatted at 2026-09-29 19:10:30 on dist-test-slave-wftq"
server_key: "ed17c9a03570527200eadfe8b1ca7d03"
server_key_iv: "aba21b7194d9524f771c395c8bdb97bd"
server_key_version: "encryptionkey@0"
I20260929 19:10:30.255534 15574 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:10:30.256330 15588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:30.256505 15574 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:30.256588 15574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "47a5c2e0b45d49cba55a3b081516c558"
format_stamp: "Formatted at 2026-09-29 19:10:30 on dist-test-slave-wftq"
server_key: "ed17c9a03570527200eadfe8b1ca7d03"
server_key_iv: "aba21b7194d9524f771c395c8bdb97bd"
server_key_version: "encryptionkey@0"
I20260929 19:10:30.256649 15574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:30.285017 15574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:30.285411 15574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:30.285547 15574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:30.290069 15574 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:46297
I20260929 19:10:30.290133 15640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:46297 every 8 connection(s)
I20260929 19:10:30.290474 15574 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:10:30.291541 15641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:30.293887 15641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558: Bootstrap starting.
I20260929 19:10:30.294445 15641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:30.294703 15641 log.cc:824] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:30.295445 15641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558: No bootstrap required, opened a new log
I20260929 19:10:30.296964 15641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47a5c2e0b45d49cba55a3b081516c558" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46297 } }
I20260929 19:10:30.297128 15641 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:30.297163 15641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47a5c2e0b45d49cba55a3b081516c558, State: Initialized, Role: FOLLOWER
I20260929 19:10:30.297312 15641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47a5c2e0b45d49cba55a3b081516c558" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46297 } }
I20260929 19:10:30.297401 15641 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:10:30.297443 15641 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:10:30.297494 15641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:30.297583 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 15574
I20260929 19:10:30.297675 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:10:30.297870 10160 external_mini_cluster.cc:1428] Setting key c73de38a1f5a78582ac0f5c29be05729
I20260929 19:10:30.298131 15641 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47a5c2e0b45d49cba55a3b081516c558" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46297 } }
I20260929 19:10:30.298226 15641 leader_election.cc:302] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47a5c2e0b45d49cba55a3b081516c558; no voters:
I20260929 19:10:30.298427 15641 leader_election.cc:288] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:10:30.298471 15645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:30.298681 15641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:30.298684 15645 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558 [term 1 LEADER]: Becoming Leader. State: Replica: 47a5c2e0b45d49cba55a3b081516c558, State: Running, Role: LEADER
I20260929 19:10:30.298840 15645 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47a5c2e0b45d49cba55a3b081516c558" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46297 } }
I20260929 19:10:30.299227 15645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47a5c2e0b45d49cba55a3b081516c558. Latest consensus state: current_term: 1 leader_uuid: "47a5c2e0b45d49cba55a3b081516c558" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47a5c2e0b45d49cba55a3b081516c558" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46297 } } }
I20260929 19:10:30.299305 15645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:30.299562 15648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47a5c2e0b45d49cba55a3b081516c558" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47a5c2e0b45d49cba55a3b081516c558" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46297 } } }
I20260929 19:10:30.299643 15648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:30.299759 15655 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:30.300287 15655 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:10:30.302032 15655 catalog_manager.cc:1347] Generated new cluster ID: 467fba1871474cf082b756d512ecf6dc
I20260929 19:10:30.302091 15655 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:10:30.304170 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:30.298421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47368 (local address 127.9.236.62:46297)
0929 19:10:30.298718 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:30.298730 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:30.298843 (+ 113us) server_negotiation.cc:407] Connection header received
0929 19:10:30.299020 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:30.299029 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:30.299123 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:30.299299 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:30.299518 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:30.300944 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:30.302036 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:30.302634 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:30.302754 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:30.303428 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:30.303431 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:30.303432 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:10:30.303650 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:30.303691 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:10:30.303704 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:30.303764 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:30.303867 (+ 103us) server_negotiation.cc:299] Negotiation successful
0929 19:10:30.303944 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":87,"threads_started":1}
I20260929 19:10:30.306231 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:10:30.311954 15655 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:10:30.312722 15655 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:30.317979 15655 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47a5c2e0b45d49cba55a3b081516c558: Generated new TSK 0
I20260929 19:10:30.318149 15655 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:10:30.400543 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:30.323874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47374 (local address 127.9.236.62:46297)
0929 19:10:30.323921 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:30.323923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:30.324059 (+ 136us) server_negotiation.cc:407] Connection header received
0929 19:10:30.324138 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:30.324139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:30.324152 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:30.324233 (+ 81us) server_negotiation.cc:226] Negotiated authn=JWT
0929 19:10:30.324372 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:30.324942 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:30.325777 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:30.326002 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:30.326056 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:30.326227 (+ 171us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 19:10:30.400214 (+ 73987us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 19:10:30.400447 (+ 233us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47374: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":25}
W20260929 19:10:30.400635 15670 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:30.323823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46297 (local address 127.0.0.1:47374)
0929 19:10:30.323985 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:30.323995 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:30.324079 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:30.324236 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:30.324237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:30.324239 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0929 19:10:30.324313 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:30.324315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:30.325036 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:30.325037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:30.325702 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:30.325705 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:30.326069 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:30.326070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:30.326129 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:30.326135 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 19:10:30.400384 (+ 74249us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 19:10:30.400530 (+ 146us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.236.62:46297: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":108,"threads_started":1}
W20260929 19:10:30.400687 15646 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47374: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260929 19:10:30.400887 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 15574
2026-09-29T19:10:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (273 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (89 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (92 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:10:30 dist-test-slave-wftq krb5kdc[15682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:10:30 dist-test-slave-wftq krb5kdc[15682](info): set up 2 sockets
Sep 29 19:10:30 dist-test-slave-wftq krb5kdc[15682](info): commencing operation
krb5kdc: starting...
W20260929 19:10:31.529016 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:10:31 dist-test-slave-wftq krb5kdc[15682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:10:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:10:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:10:31.650578 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:36473
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:39193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:36473
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:31.738440 15695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:31.738678 15695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:31.738709 15695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:31.739909 15695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:31.739949 15695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:31.739969 15695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:31.739984 15695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:31.740000 15695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:31.741493 15695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:39193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:36473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:36473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:31.741750 15695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:31.741961 15695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:31.747030 15700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:31.747143 15695 server_base.cc:1034] running on GCE node
W20260929 19:10:31.747046 15701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:31.747051 15703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:31.747613 15695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:31.747941 15695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:31.749094 15695 hybrid_clock.cc:648] HybridClock initialized: now 1790709031749077 us; error 33 us; skew 500 ppm
Sep 29 19:10:31 dist-test-slave-wftq krb5kdc[15682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709031, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:31.751612 15695 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:10:31.752099 15695 webserver.cc:460] Webserver started at http://127.9.236.62:34925/ using document root <none> and password file <none>
I20260929 19:10:31.752327 15695 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:31.752375 15695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:31.752497 15695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:31.753644 15695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "9cfde43237ca45bc8bc610f38d1d10c9"
format_stamp: "Formatted at 2026-09-29 19:10:31 on dist-test-slave-wftq"
server_key: "b5b81700d08498ac212e6bde5da167fa"
server_key_iv: "c1b5abf4aa48085776f11decc46ea1cc"
server_key_version: "encryptionkey@0"
I20260929 19:10:31.753995 15695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "9cfde43237ca45bc8bc610f38d1d10c9"
format_stamp: "Formatted at 2026-09-29 19:10:31 on dist-test-slave-wftq"
server_key: "b5b81700d08498ac212e6bde5da167fa"
server_key_iv: "c1b5abf4aa48085776f11decc46ea1cc"
server_key_version: "encryptionkey@0"
I20260929 19:10:31.755319 15695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:10:31.756122 15710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:31.756319 15695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:31.756395 15695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "9cfde43237ca45bc8bc610f38d1d10c9"
format_stamp: "Formatted at 2026-09-29 19:10:31 on dist-test-slave-wftq"
server_key: "b5b81700d08498ac212e6bde5da167fa"
server_key_iv: "c1b5abf4aa48085776f11decc46ea1cc"
server_key_version: "encryptionkey@0"
I20260929 19:10:31.756457 15695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:31.770627 15695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:31.772367 15695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:31.772498 15695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:31.776942 15695 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:36473
I20260929 19:10:31.777000 15762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:36473 every 8 connection(s)
I20260929 19:10:31.777359 15695 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:10:31.778249 15763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:31.780534 15763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9: Bootstrap starting.
I20260929 19:10:31.781071 15763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:31.781366 15763 log.cc:824] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:31.782029 15763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9: No bootstrap required, opened a new log
I20260929 19:10:31.783566 15763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cfde43237ca45bc8bc610f38d1d10c9" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36473 } }
I20260929 19:10:31.783700 15763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:31.783721 15763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cfde43237ca45bc8bc610f38d1d10c9, State: Initialized, Role: FOLLOWER
I20260929 19:10:31.783861 15763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cfde43237ca45bc8bc610f38d1d10c9" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36473 } }
I20260929 19:10:31.783921 15763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:10:31.783944 15763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:10:31.783993 15763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:31.784495 15763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cfde43237ca45bc8bc610f38d1d10c9" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36473 } }
I20260929 19:10:31.784583 15763 leader_election.cc:302] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9cfde43237ca45bc8bc610f38d1d10c9; no voters:
I20260929 19:10:31.784752 15763 leader_election.cc:288] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:10:31.784818 15766 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:31.785006 15763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:31.785228 15766 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9 [term 1 LEADER]: Becoming Leader. State: Replica: 9cfde43237ca45bc8bc610f38d1d10c9, State: Running, Role: LEADER
I20260929 19:10:31.785344 15766 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cfde43237ca45bc8bc610f38d1d10c9" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36473 } }
I20260929 19:10:31.785781 15768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cfde43237ca45bc8bc610f38d1d10c9. Latest consensus state: current_term: 1 leader_uuid: "9cfde43237ca45bc8bc610f38d1d10c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cfde43237ca45bc8bc610f38d1d10c9" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36473 } } }
I20260929 19:10:31.785920 15768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:31.785758 15767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9cfde43237ca45bc8bc610f38d1d10c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cfde43237ca45bc8bc610f38d1d10c9" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36473 } } }
I20260929 19:10:31.786072 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 15695
I20260929 19:10:31.786091 15767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:31.786166 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:10:31.786361 10160 external_mini_cluster.cc:1428] Setting key 9f923d2afaaeb2860b0441f4778b4dd0
I20260929 19:10:31.786432 15775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:31.787223 15775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:10:31.788968 15775 catalog_manager.cc:1347] Generated new cluster ID: 06f1613a5f3f4ea1acf6660c1d61910f
I20260929 19:10:31.789018 15775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:10:31 dist-test-slave-wftq krb5kdc[15682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:31.797971 15785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:31.786846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45896 (local address 127.9.236.62:36473)
0929 19:10:31.787158 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:31.787168 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:31.787196 (+ 28us) server_negotiation.cc:407] Connection header received
0929 19:10:31.787300 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:31.787306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:31.787379 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:31.787500 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:31.788173 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:31.789627 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:31.790682 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:31.791354 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:31.791469 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:31.794415 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:31.794420 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:31.794420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:31.794434 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:31.796289 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:31.796707 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:31.796709 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:31.796710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:31.796805 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:31.797011 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:31.797012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:31.797012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:31.797335 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:31.797484 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:31.797597 (+ 113us) server_negotiation.cc:299] Negotiation successful
0929 19:10:31.797736 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":102,"threads_started":1}
I20260929 19:10:31.805635 15775 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:10:31.806108 15775 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:31.810055 15775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9cfde43237ca45bc8bc610f38d1d10c9: Generated new TSK 0
I20260929 19:10:31.810192 15775 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.236.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
I20260929 19:10:31.844187 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:36473
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:39193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:31.942210 15789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:31.942420 15789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:31.942442 15789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:31.943585 15789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:31.943622 15789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:31.943646 15789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:10:31.945201 15789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:39193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:36473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:31.945391 15789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:31.945577 15789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:31.950719 15797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:31.950810 15789 server_base.cc:1034] running on GCE node
W20260929 19:10:31.950700 15795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:31.950706 15794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:31.951210 15789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:31.951463 15789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:31.952600 15789 hybrid_clock.cc:648] HybridClock initialized: now 1790709031952586 us; error 35 us; skew 500 ppm
Sep 29 19:10:31 dist-test-slave-wftq krb5kdc[15682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709031, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:31.955188 15789 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:10:31.955600 15789 webserver.cc:460] Webserver started at http://127.9.236.1:46367/ using document root <none> and password file <none>
I20260929 19:10:31.955785 15789 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:31.955826 15789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:31.955921 15789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:31.956946 15789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "ccf212c9f9fa40d38f46988072ea63d0"
format_stamp: "Formatted at 2026-09-29 19:10:31 on dist-test-slave-wftq"
server_key: "722a387c394f9584af71706cded51e18"
server_key_iv: "54271a5b19da6b478493f6ee67b06992"
server_key_version: "encryptionkey@0"
I20260929 19:10:31.957329 15789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "ccf212c9f9fa40d38f46988072ea63d0"
format_stamp: "Formatted at 2026-09-29 19:10:31 on dist-test-slave-wftq"
server_key: "722a387c394f9584af71706cded51e18"
server_key_iv: "54271a5b19da6b478493f6ee67b06992"
server_key_version: "encryptionkey@0"
I20260929 19:10:31.958583 15789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:31.959302 15804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:31.959525 15789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:31.959612 15789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "ccf212c9f9fa40d38f46988072ea63d0"
format_stamp: "Formatted at 2026-09-29 19:10:31 on dist-test-slave-wftq"
server_key: "722a387c394f9584af71706cded51e18"
server_key_iv: "54271a5b19da6b478493f6ee67b06992"
server_key_version: "encryptionkey@0"
I20260929 19:10:31.959692 15789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:31.974715 15789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:31.976761 15789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:31.976897 15789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:31.977198 15789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:31.977545 15789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:31.977581 15789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:31.977604 15789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:31.977629 15789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:31.984534 15789 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:35783
I20260929 19:10:31.984639 15917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:35783 every 8 connection(s)
I20260929 19:10:31.984931 15789 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:10:31.990391 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 15789
I20260929 19:10:31.990500 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:10:31.990692 10160 external_mini_cluster.cc:1428] Setting key 580012561365bfae855b5a46f4ff3432
Sep 29 19:10:31 dist-test-slave-wftq krb5kdc[15682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709031, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:31.994830 15785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:31.985758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:59301 (local address 127.9.236.62:36473)
0929 19:10:31.985825 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:31.985828 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:31.986095 (+ 267us) server_negotiation.cc:407] Connection header received
0929 19:10:31.987091 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:31.987092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:31.987109 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:31.987159 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:31.987801 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:31.988316 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:31.989334 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:31.989547 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:31.989600 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:31.992047 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:31.992049 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:31.992049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:31.992057 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:31.993362 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:31.993855 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:31.993858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:31.993858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:31.993926 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:31.994148 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:31.994149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:31.994149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:31.994278 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:31.994338 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:31.994610 (+ 272us) server_negotiation.cc:299] Negotiation successful
0929 19:10:31.994729 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 19:10:31.995002 15920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:31.985709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:36473 (local address 127.9.236.1:59301)
0929 19:10:31.985925 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:31.985974 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:31.986986 (+ 1012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:31.987231 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:31.987236 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:31.987520 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:31.987714 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:31.987718 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:31.988395 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:31.988396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:31.989258 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:31.989260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:31.989613 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:31.989614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:31.989758 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:31.990255 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:31.990264 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:31.991952 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:31.993445 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:31.993453 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:31.993455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:31.993782 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:31.993983 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:31.993983 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:31.993984 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:31.994092 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:31.994334 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:31.994341 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:31.994490 (+ 149us) client_negotiation.cc:769] Sending connection context
0929 19:10:31.994583 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 19:10:31.994765 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":96,"threads_started":1}
I20260929 19:10:31.995558 15918 heartbeater.cc:347] Connected to a master server at 127.9.236.62:36473
I20260929 19:10:31.995687 15918 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:31.995905 15918 heartbeater.cc:511] Master 127.9.236.62:36473 requested a full tablet report, sending...
I20260929 19:10:31.996486 15727 ts_manager.cc:194] Registered new tserver with Master: ccf212c9f9fa40d38f46988072ea63d0 (127.9.236.1:35783)
I20260929 19:10:31.997198 15727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:59301
WARNING: no policy specified for kudu/127.9.236.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
I20260929 19:10:32.016167 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:36473
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:39193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:32.097901 15923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:32.098099 15923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:32.098120 15923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:32.099195 15923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:32.099232 15923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:32.099255 15923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:10:32.100661 15923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:39193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:36473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:32.100854 15923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:32.101052 15923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:32.106285 15931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:32.106362 15929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:32.106312 15928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:32.106516 15923 server_base.cc:1034] running on GCE node
I20260929 19:10:32.106760 15923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:32.107036 15923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:32.108211 15923 hybrid_clock.cc:648] HybridClock initialized: now 1790709032108179 us; error 48 us; skew 500 ppm
Sep 29 19:10:32 dist-test-slave-wftq krb5kdc[15682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709032, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:32.110952 15923 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:10:32.111433 15923 webserver.cc:460] Webserver started at http://127.9.236.2:36905/ using document root <none> and password file <none>
I20260929 19:10:32.111649 15923 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:32.111694 15923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:32.111824 15923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:32.112982 15923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "b59099a4c0754c4c90732b7090e63f26"
format_stamp: "Formatted at 2026-09-29 19:10:32 on dist-test-slave-wftq"
server_key: "e929abb5f8e1723150bc987f0274f27b"
server_key_iv: "0a21c40cbc96acf7609121e726f7121f"
server_key_version: "encryptionkey@0"
I20260929 19:10:32.113445 15923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "b59099a4c0754c4c90732b7090e63f26"
format_stamp: "Formatted at 2026-09-29 19:10:32 on dist-test-slave-wftq"
server_key: "e929abb5f8e1723150bc987f0274f27b"
server_key_iv: "0a21c40cbc96acf7609121e726f7121f"
server_key_version: "encryptionkey@0"
I20260929 19:10:32.114873 15923 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:32.115720 15938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:32.115921 15923 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:32.116022 15923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "b59099a4c0754c4c90732b7090e63f26"
format_stamp: "Formatted at 2026-09-29 19:10:32 on dist-test-slave-wftq"
server_key: "e929abb5f8e1723150bc987f0274f27b"
server_key_iv: "0a21c40cbc96acf7609121e726f7121f"
server_key_version: "encryptionkey@0"
I20260929 19:10:32.116092 15923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:32.127915 15923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:32.130137 15923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:32.130278 15923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:32.130535 15923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:32.130856 15923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:32.130888 15923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:32.130918 15923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:32.130939 15923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:32.137581 15923 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:45399
I20260929 19:10:32.137672 16051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:45399 every 8 connection(s)
I20260929 19:10:32.137953 15923 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:10:32.141089 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 15923
I20260929 19:10:32.141212 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:10:32.141390 10160 external_mini_cluster.cc:1428] Setting key c303819fd2cb581b7a96b255285ed851
Sep 29 19:10:32 dist-test-slave-wftq krb5kdc[15682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709032, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:32.148432 15785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:32.138850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:44539 (local address 127.9.236.62:36473)
0929 19:10:32.138909 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:32.138911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:32.139042 (+ 131us) server_negotiation.cc:407] Connection header received
0929 19:10:32.140028 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:32.140029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:32.140040 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:32.140131 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:32.140758 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:32.141357 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:32.142609 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:32.142806 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:32.142865 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:32.145507 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:32.145516 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:32.145516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:32.145525 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:32.146973 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:32.147479 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:32.147480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:32.147480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:32.147534 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:32.147801 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:32.147801 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:32.147802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:32.147902 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:32.147951 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:32.148266 (+ 315us) server_negotiation.cc:299] Negotiation successful
0929 19:10:32.148361 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 19:10:32.148703 16054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:32.138719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:36473 (local address 127.9.236.2:44539)
0929 19:10:32.138901 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:32.138944 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:32.139906 (+ 962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:32.140194 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:32.140198 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:32.140456 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:32.140670 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:32.140675 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:32.141451 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:32.141452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:32.142519 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:32.142523 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:32.142886 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:32.142887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:32.143033 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:32.143659 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:32.143669 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:32.145401 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:32.147073 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:32.147080 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:32.147083 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:32.147396 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:32.147601 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:32.147603 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:32.147603 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:32.147736 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:32.147964 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:32.147969 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:32.148132 (+ 163us) client_negotiation.cc:769] Sending connection context
0929 19:10:32.148257 (+ 125us) client_negotiation.cc:240] Negotiation successful
0929 19:10:32.148435 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260929 19:10:32.149322 16052 heartbeater.cc:347] Connected to a master server at 127.9.236.62:36473
I20260929 19:10:32.149434 16052 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:32.149637 16052 heartbeater.cc:511] Master 127.9.236.62:36473 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.236.3@KRBTEST.COM; defaulting to no policy
I20260929 19:10:32.150072 15727 ts_manager.cc:194] Registered new tserver with Master: b59099a4c0754c4c90732b7090e63f26 (127.9.236.2:45399)
I20260929 19:10:32.150477 15727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:44539
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
I20260929 19:10:32.168583 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:36473
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:39193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:32.257345 16057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:32.257575 16057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:32.257601 16057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:32.258795 16057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:32.258836 16057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:32.258863 16057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:10:32.260388 16057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:39193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:36473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:32.260625 16057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:32.260849 16057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:32.266304 16065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:32.266405 16057 server_base.cc:1034] running on GCE node
W20260929 19:10:32.266279 16063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:32.266279 16062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:32.266799 16057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:32.267067 16057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:32.268230 16057 hybrid_clock.cc:648] HybridClock initialized: now 1790709032268204 us; error 36 us; skew 500 ppm
Sep 29 19:10:32 dist-test-slave-wftq krb5kdc[15682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709032, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:32.271157 16057 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:10:32.271652 16057 webserver.cc:460] Webserver started at http://127.9.236.3:35797/ using document root <none> and password file <none>
I20260929 19:10:32.271852 16057 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:32.271899 16057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:32.272011 16057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:32.273191 16057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "23b2e37dc57745aca740373406e75cb9"
format_stamp: "Formatted at 2026-09-29 19:10:32 on dist-test-slave-wftq"
server_key: "93539925150e206d2da579e60aec5f82"
server_key_iv: "bdf954823161e618123a62300fb42ee8"
server_key_version: "encryptionkey@0"
I20260929 19:10:32.273626 16057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "23b2e37dc57745aca740373406e75cb9"
format_stamp: "Formatted at 2026-09-29 19:10:32 on dist-test-slave-wftq"
server_key: "93539925150e206d2da579e60aec5f82"
server_key_iv: "bdf954823161e618123a62300fb42ee8"
server_key_version: "encryptionkey@0"
I20260929 19:10:32.275018 16057 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 19:10:32.275784 16072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:32.275982 16057 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:32.276080 16057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "23b2e37dc57745aca740373406e75cb9"
format_stamp: "Formatted at 2026-09-29 19:10:32 on dist-test-slave-wftq"
server_key: "93539925150e206d2da579e60aec5f82"
server_key_iv: "bdf954823161e618123a62300fb42ee8"
server_key_version: "encryptionkey@0"
I20260929 19:10:32.276165 16057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:32.297838 16057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:32.300084 16057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:32.300207 16057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:32.300448 16057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:32.300776 16057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:32.300815 16057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:32.300837 16057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:32.300858 16057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:32.307232 16057 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:39939
I20260929 19:10:32.307319 16185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:39939 every 8 connection(s)
I20260929 19:10:32.307628 16057 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:10:32.314098 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 16057
I20260929 19:10:32.314186 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:10:32.314363 10160 external_mini_cluster.cc:1428] Setting key b979b30f3f240a47078f53cc20c675a8
Sep 29 19:10:32 dist-test-slave-wftq krb5kdc[15682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709032, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:32.317478 15785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:32.308559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:43487 (local address 127.9.236.62:36473)
0929 19:10:32.308656 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:32.308660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:32.308823 (+ 163us) server_negotiation.cc:407] Connection header received
0929 19:10:32.309904 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:32.309906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:32.309920 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:32.309982 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:32.310716 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:32.311255 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:32.312298 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:32.312492 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:32.312549 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:32.314966 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:32.314968 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:32.314968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:32.314975 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:32.316194 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:32.316566 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:32.316566 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:32.316566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:32.316610 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:32.316818 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:32.316819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:32.316819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:32.316961 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:32.317015 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:32.317341 (+ 326us) server_negotiation.cc:299] Negotiation successful
0929 19:10:32.317409 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 19:10:32.317649 16188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:32.308447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:36473 (local address 127.9.236.3:43487)
0929 19:10:32.308661 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:32.308717 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:32.309783 (+ 1066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:32.310061 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:32.310066 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:32.310382 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:32.310612 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:32.310617 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:32.311337 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:32.311339 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:32.312211 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:32.312214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:32.312561 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:32.312562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:32.312702 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:32.313267 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:32.313277 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:32.314869 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:32.316269 (+ 1400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:32.316275 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:32.316278 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:32.316501 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:32.316665 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:32.316665 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:32.316665 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:32.316760 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:32.317015 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:32.317018 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:32.317193 (+ 175us) client_negotiation.cc:769] Sending connection context
0929 19:10:32.317288 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 19:10:32.317426 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":95,"threads_started":1}
I20260929 19:10:32.318159 16186 heartbeater.cc:347] Connected to a master server at 127.9.236.62:36473
I20260929 19:10:32.318255 16186 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:32.318423 16186 heartbeater.cc:511] Master 127.9.236.62:36473 requested a full tablet report, sending...
I20260929 19:10:32.318763 15727 ts_manager.cc:194] Registered new tserver with Master: 23b2e37dc57745aca740373406e75cb9 (127.9.236.3:39939)
I20260929 19:10:32.319188 15727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:43487
I20260929 19:10:32.326457 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/test-admin.keytab.
Sep 29 19:10:32 dist-test-slave-wftq krb5kdc[15682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:32.342254 10160 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260929 19:10:32.343912 16193 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:32.343136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49042 (local address 127.9.236.1:35783)
0929 19:10:32.343333 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:32.343339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:32.343348 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:32.343372 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:32.343374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:32.343388 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:32.343453 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:32.343837 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49042: BlockingRecv error: recv got EOF from 127.0.0.1:49042 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":85,"threads_started":1}
W20260929 19:10:32.344061 10160 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/krb5cc' of type FILE
Sep 29 19:10:32 dist-test-slave-wftq krb5kdc[15682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:32.345220 10160 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 19:10:32 dist-test-slave-wftq krb5kdc[15682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:32.346042 10160 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 19:10:32 dist-test-slave-wftq krb5kdc[15682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.1@KRBTEST.COM
I20260929 19:10:32.354302 16193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:32.346580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49044 (local address 127.9.236.1:35783)
0929 19:10:32.346660 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:32.346662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:32.346709 (+ 47us) server_negotiation.cc:407] Connection header received
0929 19:10:32.346771 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:32.346771 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:32.346786 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:32.346841 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:32.347244 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:32.348094 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:32.348853 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:32.349288 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:32.349351 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:32.351538 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:32.351548 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:32.351549 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:32.351556 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:32.353083 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:32.353535 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:32.353538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:32.353539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:32.353595 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:32.353749 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:32.353750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:32.353750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:32.353878 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:32.354001 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:32.354119 (+ 118us) server_negotiation.cc:299] Negotiation successful
0929 19:10:32.354189 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260929 19:10:32.356102 16193 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:32.355579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49060 (local address 127.9.236.1:35783)
0929 19:10:32.355648 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:32.355650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:32.355689 (+ 39us) server_negotiation.cc:407] Connection header received
0929 19:10:32.355751 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:32.355751 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:32.355761 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:32.355808 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:32.356059 (+ 251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49060: BlockingRecv error: recv got EOF from 127.0.0.1:49060 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260929 19:10:32.356290 10160 init.cc:244] Error while opening credential cache '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/krb5cc)
Sep 29 19:10:32 dist-test-slave-wftq krb5kdc[15682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:32.357421 10160 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 19:10:32 dist-test-slave-wftq krb5kdc[15682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:32.358258 10160 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 19:10:32 dist-test-slave-wftq krb5kdc[15682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.1@KRBTEST.COM
I20260929 19:10:32.365702 16193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:32.358849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49070 (local address 127.9.236.1:35783)
0929 19:10:32.358902 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:32.358904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:32.358979 (+ 75us) server_negotiation.cc:407] Connection header received
0929 19:10:32.359037 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:32.359038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:32.359046 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:32.359088 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:32.359503 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:32.360068 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:32.360939 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:32.361228 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:32.361289 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:32.363566 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:32.363567 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:32.363568 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:32.363574 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:32.364773 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:32.365082 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:32.365083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:32.365083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:32.365161 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:32.365311 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:32.365311 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:32.365311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:32.365400 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:32.365474 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:32.365572 (+ 98us) server_negotiation.cc:299] Negotiation successful
0929 19:10:32.365637 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260929 19:10:32.367269 16193 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:32.366730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49086 (local address 127.9.236.1:35783)
0929 19:10:32.366785 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:32.366786 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:32.366822 (+ 36us) server_negotiation.cc:407] Connection header received
0929 19:10:32.366884 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:32.366885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:32.366895 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:32.366940 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:32.367208 (+ 268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49086: BlockingRecv error: recv got EOF from 127.0.0.1:49086 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260929 19:10:32.367352 10160 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790709008521303-10160-0/krb5kdc/krb5cc' of type FILE
Sep 29 19:10:32 dist-test-slave-wftq krb5kdc[15682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:32.368285 10160 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 19:10:32 dist-test-slave-wftq krb5kdc[15682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:32.369081 10160 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 19:10:32 dist-test-slave-wftq krb5kdc[15682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.1@KRBTEST.COM
I20260929 19:10:32.375765 16193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:32.369663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49102 (local address 127.9.236.1:35783)
0929 19:10:32.369702 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:32.369703 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:32.369713 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:10:32.369767 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:32.369768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:32.369793 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:32.369844 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:32.370232 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:32.370708 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:32.371468 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:32.371647 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:32.371696 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:32.373742 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:32.373743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:32.373744 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:32.373749 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:32.374920 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:32.375214 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:32.375215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:32.375215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:32.375258 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:32.375398 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:32.375399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:32.375399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:32.375487 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:32.375558 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:32.375630 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 19:10:32.375680 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 19:10:32.376457 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 15789
I20260929 19:10:32.381933 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 15923
I20260929 19:10:32.386507 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 16057
I20260929 19:10:32.390313 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 15695
2026-09-29T19:10:32Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1809 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:10:32 dist-test-slave-wftq krb5kdc[16206](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:10:32 dist-test-slave-wftq krb5kdc[16206](info): set up 2 sockets
Sep 29 19:10:32 dist-test-slave-wftq krb5kdc[16206](info): commencing operation
krb5kdc: starting...
W20260929 19:10:33.357594 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:10:33 dist-test-slave-wftq krb5kdc[16206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:10:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:10:33Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.62.keytab.
Entry for principal oryx/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.62.keytab.
I20260929 19:10:33.480737 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:38041
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.62.keytab
--principal=oryx/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:38041
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:33.562812 16219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:33.563022 16219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:33.563042 16219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:33.564126 16219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:33.564163 16219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:33.564177 16219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:33.564188 16219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:33.564198 16219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:33.565660 16219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:38041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.62.keytab
--principal=oryx/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:38041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:33.566035 16219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:33.566323 16219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:33.571123 16227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:33.571228 16224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:33.571228 16225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:33.571413 16219 server_base.cc:1034] running on GCE node
I20260929 19:10:33.571749 16219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:33.572049 16219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:33.573165 16219 hybrid_clock.cc:648] HybridClock initialized: now 1790709033573142 us; error 48 us; skew 500 ppm
Sep 29 19:10:33 dist-test-slave-wftq krb5kdc[16206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709033, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:33.575537 16219 init.cc:377] Logged in from keytab as oryx/127.9.236.62@KRBTEST.COM (short username oryx)
I20260929 19:10:33.575963 16219 webserver.cc:460] Webserver started at http://127.9.236.62:45505/ using document root <none> and password file <none>
I20260929 19:10:33.576166 16219 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:33.576211 16219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:33.576327 16219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:33.577345 16219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "64eaa0faf742494680dadbf96bb413f0"
format_stamp: "Formatted at 2026-09-29 19:10:33 on dist-test-slave-wftq"
server_key: "9f1eea28d48b002496411ea1c9697ad5"
server_key_iv: "f63073fd35f946c3559c6e0caf648b5d"
server_key_version: "encryptionkey@0"
I20260929 19:10:33.577646 16219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "64eaa0faf742494680dadbf96bb413f0"
format_stamp: "Formatted at 2026-09-29 19:10:33 on dist-test-slave-wftq"
server_key: "9f1eea28d48b002496411ea1c9697ad5"
server_key_iv: "f63073fd35f946c3559c6e0caf648b5d"
server_key_version: "encryptionkey@0"
I20260929 19:10:33.578874 16219 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:33.579587 16234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:33.579795 16219 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:33.579879 16219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "64eaa0faf742494680dadbf96bb413f0"
format_stamp: "Formatted at 2026-09-29 19:10:33 on dist-test-slave-wftq"
server_key: "9f1eea28d48b002496411ea1c9697ad5"
server_key_iv: "f63073fd35f946c3559c6e0caf648b5d"
server_key_version: "encryptionkey@0"
I20260929 19:10:33.579937 16219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:33.602979 16219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:33.605083 16219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:33.605233 16219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:33.609176 16219 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:38041
I20260929 19:10:33.609236 16286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:38041 every 8 connection(s)
I20260929 19:10:33.609556 16219 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:10:33.610723 16287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:33.613454 16287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0: Bootstrap starting.
I20260929 19:10:33.614079 16287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:33.614359 16287 log.cc:824] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:33.615123 16287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0: No bootstrap required, opened a new log
I20260929 19:10:33.615923 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 16219
I20260929 19:10:33.616008 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:10:33.616173 10160 external_mini_cluster.cc:1428] Setting key b534c002fea12a0ebc6b348be34350ff
I20260929 19:10:33.617082 16287 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } }
I20260929 19:10:33.617268 16287 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:33.617300 16287 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64eaa0faf742494680dadbf96bb413f0, State: Initialized, Role: FOLLOWER
I20260929 19:10:33.617436 16287 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } }
I20260929 19:10:33.617516 16287 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:10:33.617551 16287 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:10:33.617604 16287 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:33.618538 16287 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } }
I20260929 19:10:33.618641 16287 leader_election.cc:302] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64eaa0faf742494680dadbf96bb413f0; no voters:
I20260929 19:10:33.618816 16287 leader_election.cc:288] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:10:33.618853 16292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:33.618973 16292 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [term 1 LEADER]: Becoming Leader. State: Replica: 64eaa0faf742494680dadbf96bb413f0, State: Running, Role: LEADER
I20260929 19:10:33.619093 16287 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:33.619067 16292 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } }
I20260929 19:10:33.619436 16292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64eaa0faf742494680dadbf96bb413f0. Latest consensus state: current_term: 1 leader_uuid: "64eaa0faf742494680dadbf96bb413f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } }
I20260929 19:10:33.619510 16292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:33.619665 16294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64eaa0faf742494680dadbf96bb413f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } }
I20260929 19:10:33.619736 16294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [sys.catalog]: This master's current role is: LEADER
W20260929 19:10:33.621059 16308 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:10:33.621157 16308 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:10:33.621209 16297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:33.621693 16297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 19:10:33 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM
I20260929 19:10:33.623529 16297 catalog_manager.cc:1347] Generated new cluster ID: 5efb130dadec41a89dcc31b7161887e6
I20260929 19:10:33.623585 16297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:10:33.627578 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:33.616714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54572 (local address 127.9.236.62:38041)
0929 19:10:33.616902 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:33.616911 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:33.616940 (+ 29us) server_negotiation.cc:407] Connection header received
0929 19:10:33.617056 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:33.617062 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:33.617134 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:33.617263 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:33.617935 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:33.618881 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:33.619873 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:33.620606 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:33.620760 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:33.623719 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:33.623724 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:33.623724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:33.623738 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:33.625702 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:33.626153 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:33.626156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:33.626157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:33.626305 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:33.626506 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:33.626506 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:33.626507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:33.626815 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:33.626995 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:33.627213 (+ 218us) server_negotiation.cc:299] Negotiation successful
0929 19:10:33.627373 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":87,"threads_started":1}
I20260929 19:10:33.639307 16297 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:10:33.640067 16297 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:33.644614 16297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0: Generated new TSK 0
I20260929 19:10:33.644784 16297 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.236.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.236.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.236.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.1.keytab.
Entry for principal oryx/127.9.236.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.1.keytab.
I20260929 19:10:33.672145 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:38041
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.1.keytab
--principal=oryx/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:33.761925 16313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:33.762151 16313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:33.762188 16313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:33.763355 16313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:33.763399 16313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:33.763435 16313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:10:33.764868 16313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.1.keytab
--principal=oryx/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:38041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:33.765126 16313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:33.765343 16313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:33.770478 16318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:33.770493 16319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:33.770581 16321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:33.770833 16313 server_base.cc:1034] running on GCE node
I20260929 19:10:33.771049 16313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:33.771301 16313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:33.772445 16313 hybrid_clock.cc:648] HybridClock initialized: now 1790709033772426 us; error 34 us; skew 500 ppm
Sep 29 19:10:33 dist-test-slave-wftq krb5kdc[16206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709033, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:33.775110 16313 init.cc:377] Logged in from keytab as oryx/127.9.236.1@KRBTEST.COM (short username oryx)
I20260929 19:10:33.775557 16313 webserver.cc:460] Webserver started at http://127.9.236.1:34031/ using document root <none> and password file <none>
I20260929 19:10:33.775774 16313 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:33.775815 16313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:33.775915 16313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:33.777021 16313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "56e2d743ca3e4d018b7e997ec34773c0"
format_stamp: "Formatted at 2026-09-29 19:10:33 on dist-test-slave-wftq"
server_key: "10bb7653e6ed3c2bc1e67e1b7a447c7b"
server_key_iv: "3869670918a7a76833a0e1e67a4832e9"
server_key_version: "encryptionkey@0"
I20260929 19:10:33.777442 16313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "56e2d743ca3e4d018b7e997ec34773c0"
format_stamp: "Formatted at 2026-09-29 19:10:33 on dist-test-slave-wftq"
server_key: "10bb7653e6ed3c2bc1e67e1b7a447c7b"
server_key_iv: "3869670918a7a76833a0e1e67a4832e9"
server_key_version: "encryptionkey@0"
I20260929 19:10:33.778832 16313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:33.779726 16328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:33.779950 16313 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:33.780063 16313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "56e2d743ca3e4d018b7e997ec34773c0"
format_stamp: "Formatted at 2026-09-29 19:10:33 on dist-test-slave-wftq"
server_key: "10bb7653e6ed3c2bc1e67e1b7a447c7b"
server_key_iv: "3869670918a7a76833a0e1e67a4832e9"
server_key_version: "encryptionkey@0"
I20260929 19:10:33.780130 16313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:33.792058 16313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:33.794080 16313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:33.794210 16313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:33.794427 16313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:33.794752 16313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:33.794785 16313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:33.794814 16313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:33.794842 16313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:33.801596 16313 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:46669
I20260929 19:10:33.801703 16441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:46669 every 8 connection(s)
I20260929 19:10:33.801980 16313 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:10:33.807999 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 16313
I20260929 19:10:33.808104 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:10:33.808331 10160 external_mini_cluster.cc:1428] Setting key 3a915c79ccc71601ebcc5431506e5651
Sep 29 19:10:33 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709033, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM
I20260929 19:10:33.812289 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:33.802781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:51283 (local address 127.9.236.62:38041)
0929 19:10:33.802883 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:33.802886 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:33.803077 (+ 191us) server_negotiation.cc:407] Connection header received
0929 19:10:33.803659 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:33.803661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:33.803681 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:33.803754 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:33.804766 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:33.805441 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:33.806537 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:33.806761 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:33.806829 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:33.809370 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:33.809372 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:33.809372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:33.809385 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:33.810684 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:33.811155 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:33.811156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:33.811157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:33.811249 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:33.811587 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:33.811588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:33.811589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:33.811723 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:33.811794 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:33.812118 (+ 324us) server_negotiation.cc:299] Negotiation successful
0929 19:10:33.812214 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 19:10:33.812429 16444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:33.802741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.1:51283)
0929 19:10:33.802929 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:33.802973 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:33.803514 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:33.803844 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:33.803848 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:33.804450 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:33.804661 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:33.804665 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:33.805572 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:33.805573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:33.806441 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:33.806443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:33.806847 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:33.806848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:33.806955 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:33.807559 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:33.807567 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:33.809271 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:33.810760 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:33.810767 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:33.810770 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:33.811061 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:33.811330 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:33.811331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:33.811331 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:33.811464 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:33.811799 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:33.811804 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:33.811974 (+ 170us) client_negotiation.cc:769] Sending connection context
0929 19:10:33.812095 (+ 121us) client_negotiation.cc:240] Negotiation successful
0929 19:10:33.812234 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":100,"threads_started":1}
I20260929 19:10:33.813030 16442 heartbeater.cc:347] Connected to a master server at 127.9.236.62:38041
I20260929 19:10:33.813158 16442 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:33.813375 16442 heartbeater.cc:511] Master 127.9.236.62:38041 requested a full tablet report, sending...
I20260929 19:10:33.813954 16251 ts_manager.cc:194] Registered new tserver with Master: 56e2d743ca3e4d018b7e997ec34773c0 (127.9.236.1:46669)
I20260929 19:10:33.814661 16251 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:51283
WARNING: no policy specified for oryx/127.9.236.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.236.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.236.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.2.keytab.
Entry for principal oryx/127.9.236.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.2.keytab.
I20260929 19:10:33.835518 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:38041
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.2.keytab
--principal=oryx/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:33.916464 16447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:33.916661 16447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:33.916683 16447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:33.917836 16447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:33.917878 16447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:33.917904 16447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:10:33.919355 16447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.2.keytab
--principal=oryx/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:38041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:33.919565 16447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:33.919809 16447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:33.924850 16452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:33.924850 16455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:33.924887 16453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:33.924988 16447 server_base.cc:1034] running on GCE node
I20260929 19:10:33.925369 16447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:33.925653 16447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:33.926810 16447 hybrid_clock.cc:648] HybridClock initialized: now 1790709033926793 us; error 38 us; skew 500 ppm
Sep 29 19:10:33 dist-test-slave-wftq krb5kdc[16206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709033, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:33.929654 16447 init.cc:377] Logged in from keytab as oryx/127.9.236.2@KRBTEST.COM (short username oryx)
I20260929 19:10:33.930188 16447 webserver.cc:460] Webserver started at http://127.9.236.2:44295/ using document root <none> and password file <none>
I20260929 19:10:33.930411 16447 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:33.930461 16447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:33.930598 16447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:33.931761 16447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "85ed2357da7247fc89f778b65e83aa92"
format_stamp: "Formatted at 2026-09-29 19:10:33 on dist-test-slave-wftq"
server_key: "6c1e5f755cf41e8a098b39ae2971af6b"
server_key_iv: "9fcade02521f44ccaf658ad5e63aa65d"
server_key_version: "encryptionkey@0"
I20260929 19:10:33.932116 16447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "85ed2357da7247fc89f778b65e83aa92"
format_stamp: "Formatted at 2026-09-29 19:10:33 on dist-test-slave-wftq"
server_key: "6c1e5f755cf41e8a098b39ae2971af6b"
server_key_iv: "9fcade02521f44ccaf658ad5e63aa65d"
server_key_version: "encryptionkey@0"
I20260929 19:10:33.933533 16447 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:10:33.934348 16462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:33.934573 16447 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:33.934662 16447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "85ed2357da7247fc89f778b65e83aa92"
format_stamp: "Formatted at 2026-09-29 19:10:33 on dist-test-slave-wftq"
server_key: "6c1e5f755cf41e8a098b39ae2971af6b"
server_key_iv: "9fcade02521f44ccaf658ad5e63aa65d"
server_key_version: "encryptionkey@0"
I20260929 19:10:33.934724 16447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:33.954177 16447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:33.956429 16447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:33.956574 16447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:33.956820 16447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:33.957168 16447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:33.957206 16447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:33.957239 16447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:33.957260 16447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:33.964001 16447 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:39003
I20260929 19:10:33.964099 16575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:39003 every 8 connection(s)
I20260929 19:10:33.964398 16447 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:10:33.970721 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 16447
I20260929 19:10:33.970844 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:10:33.971043 10160 external_mini_cluster.cc:1428] Setting key 4634755f76de34a023a11384035b8541
Sep 29 19:10:33 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709033, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM
I20260929 19:10:33.974529 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:33.965215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:60793 (local address 127.9.236.62:38041)
0929 19:10:33.965318 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:33.965321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:33.965487 (+ 166us) server_negotiation.cc:407] Connection header received
0929 19:10:33.966108 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:33.966109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:33.966121 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:33.966185 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:33.967324 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:33.967915 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:33.968957 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:33.969196 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:33.969251 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:33.971620 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:33.971622 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:33.971622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:33.971630 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:33.972926 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:33.973450 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:33.973452 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:33.973452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:33.973520 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:33.973835 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:33.973837 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:33.973837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:33.973978 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:33.974016 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:33.974322 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 19:10:33.974425 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 19:10:33.974686 16578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:33.965150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.2:60793)
0929 19:10:33.965345 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:33.965386 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:33.965972 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:33.966272 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:33.966276 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:33.966949 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:33.967190 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:33.967195 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:33.967991 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:33.967991 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:33.968872 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:33.968874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:33.969269 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:33.969270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:33.969380 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:33.969882 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:33.969891 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:33.971514 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:33.973014 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:33.973021 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:33.973024 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:33.973362 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:33.973591 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:33.973592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:33.973592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:33.973740 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:33.974052 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:33.974057 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:33.974193 (+ 136us) client_negotiation.cc:769] Sending connection context
0929 19:10:33.974284 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 19:10:33.974453 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":101,"threads_started":1}
I20260929 19:10:33.975193 16576 heartbeater.cc:347] Connected to a master server at 127.9.236.62:38041
I20260929 19:10:33.975291 16576 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:33.975478 16576 heartbeater.cc:511] Master 127.9.236.62:38041 requested a full tablet report, sending...
I20260929 19:10:33.975889 16251 ts_manager.cc:194] Registered new tserver with Master: 85ed2357da7247fc89f778b65e83aa92 (127.9.236.2:39003)
I20260929 19:10:33.976279 16251 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:60793
WARNING: no policy specified for oryx/127.9.236.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.236.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.236.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.3.keytab.
Entry for principal oryx/127.9.236.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.3.keytab.
I20260929 19:10:33.996978 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:38041
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.3.keytab
--principal=oryx/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:34.077761 16581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:34.077977 16581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:34.077999 16581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:34.079077 16581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:34.079116 16581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:34.079141 16581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:10:34.080533 16581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.3.keytab
--principal=oryx/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:38041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:34.080711 16581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:34.080888 16581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:34.086455 16587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:34.086489 16586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:34.086491 16589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:34.086699 16581 server_base.cc:1034] running on GCE node
I20260929 19:10:34.086920 16581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:34.087176 16581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:34.088325 16581 hybrid_clock.cc:648] HybridClock initialized: now 1790709034088309 us; error 37 us; skew 500 ppm
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709034, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:34.090934 16581 init.cc:377] Logged in from keytab as oryx/127.9.236.3@KRBTEST.COM (short username oryx)
I20260929 19:10:34.091414 16581 webserver.cc:460] Webserver started at http://127.9.236.3:32825/ using document root <none> and password file <none>
I20260929 19:10:34.091617 16581 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:34.091679 16581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:34.091801 16581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:34.092819 16581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "4bff726d322940009260aeb6fc83fa15"
format_stamp: "Formatted at 2026-09-29 19:10:34 on dist-test-slave-wftq"
server_key: "c9fa7bf04857b4ace769a42f16f5432b"
server_key_iv: "d06b20216872eaf8a6d2d9c9f7adecc6"
server_key_version: "encryptionkey@0"
I20260929 19:10:34.093161 16581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "4bff726d322940009260aeb6fc83fa15"
format_stamp: "Formatted at 2026-09-29 19:10:34 on dist-test-slave-wftq"
server_key: "c9fa7bf04857b4ace769a42f16f5432b"
server_key_iv: "d06b20216872eaf8a6d2d9c9f7adecc6"
server_key_version: "encryptionkey@0"
I20260929 19:10:34.094416 16581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:34.095134 16596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:34.095320 16581 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:34.095422 16581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "4bff726d322940009260aeb6fc83fa15"
format_stamp: "Formatted at 2026-09-29 19:10:34 on dist-test-slave-wftq"
server_key: "c9fa7bf04857b4ace769a42f16f5432b"
server_key_iv: "d06b20216872eaf8a6d2d9c9f7adecc6"
server_key_version: "encryptionkey@0"
I20260929 19:10:34.095503 16581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:34.105811 16581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:34.107834 16581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:34.107959 16581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:34.108165 16581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:34.108481 16581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:34.108515 16581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:34.108537 16581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:34.108561 16581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:34.115371 16581 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:43875
I20260929 19:10:34.115465 16709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:43875 every 8 connection(s)
I20260929 19:10:34.115760 16581 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:10:34.122205 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 16581
I20260929 19:10:34.122304 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709034, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM
I20260929 19:10:34.122469 10160 external_mini_cluster.cc:1428] Setting key e3d051da627d9e86cd438e053cdf6901
I20260929 19:10:34.125548 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.116554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:41283 (local address 127.9.236.62:38041)
0929 19:10:34.116629 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.116631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.116861 (+ 230us) server_negotiation.cc:407] Connection header received
0929 19:10:34.117451 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.117452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.117464 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.117550 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.118533 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.119081 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.120247 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.120487 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.120573 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.122913 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.122914 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.122914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.122920 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.124310 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.124747 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.124748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.124748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.124794 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.124986 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.124987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.124987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.125083 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.125175 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.125415 (+ 240us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.125481 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 19:10:34.125792 16712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.116504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.3:41283)
0929 19:10:34.116685 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.116731 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.117349 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.117609 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.117613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.118211 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:34.118432 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.118436 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.119165 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.119166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.120143 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.120145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.120581 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.120582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.120698 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.121275 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:34.121286 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:34.122822 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:34.124389 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.124396 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.124398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.124666 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.124844 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.124844 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.124845 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.124938 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.125167 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:34.125172 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:34.125300 (+ 128us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.125391 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.125551 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":107,"threads_started":1}
I20260929 19:10:34.126305 16710 heartbeater.cc:347] Connected to a master server at 127.9.236.62:38041
I20260929 19:10:34.126410 16710 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:34.126585 16710 heartbeater.cc:511] Master 127.9.236.62:38041 requested a full tablet report, sending...
I20260929 19:10:34.126935 16251 ts_manager.cc:194] Registered new tserver with Master: 4bff726d322940009260aeb6fc83fa15 (127.9.236.3:43875)
I20260929 19:10:34.127310 16251 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:41283
I20260929 19:10:34.134737 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.236.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.236.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.236.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.61.keytab.
Entry for principal oryx/127.9.236.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.61.keytab.
I20260929 19:10:34.161711 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.61:43681
--webserver_interface=127.9.236.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.61.keytab
--principal=oryx/127.9.236.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:38041,127.9.236.61:43681
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:34.242985 16715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:34.243199 16715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:34.243239 16715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:34.244375 16715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:34.244416 16715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:34.244433 16715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:34.244450 16715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:34.244464 16715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:34.245966 16715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:38041,127.9.236.61:43681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.61.keytab
--principal=oryx/127.9.236.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.61:43681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.236.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
W20260929 19:10:34.246080 16715 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.9.236.62:38041,127.9.236.61:43681'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260929 19:10:34.246253 16715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:34.246457 16715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:34.251314 16723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:34.251376 16720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:34.251390 16715 server_base.cc:1034] running on GCE node
W20260929 19:10:34.251327 16721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:34.251849 16715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:34.252134 16715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:34.253300 16715 hybrid_clock.cc:648] HybridClock initialized: now 1790709034253279 us; error 34 us; skew 500 ppm
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709034, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:34.255940 16715 init.cc:377] Logged in from keytab as oryx/127.9.236.61@KRBTEST.COM (short username oryx)
I20260929 19:10:34.256407 16715 webserver.cc:460] Webserver started at http://127.9.236.61:44199/ using document root <none> and password file <none>
I20260929 19:10:34.256623 16715 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:34.256669 16715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:34.256773 16715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:34.257812 16715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/data/instance:
uuid: "f0c0560533814a7cbebc5c5940f38a3d"
format_stamp: "Formatted at 2026-09-29 19:10:34 on dist-test-slave-wftq"
server_key: "0e5e55dd13270c1193f4575a3a235ad0"
server_key_iv: "a0910f1f8b7a43712480cf5d52a1545a"
server_key_version: "encryptionkey@0"
I20260929 19:10:34.258096 16715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/wal/instance:
uuid: "f0c0560533814a7cbebc5c5940f38a3d"
format_stamp: "Formatted at 2026-09-29 19:10:34 on dist-test-slave-wftq"
server_key: "0e5e55dd13270c1193f4575a3a235ad0"
server_key_iv: "a0910f1f8b7a43712480cf5d52a1545a"
server_key_version: "encryptionkey@0"
I20260929 19:10:34.259321 16715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:10:34.260047 16730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:34.260268 16715 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:34.260367 16715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/wal
uuid: "f0c0560533814a7cbebc5c5940f38a3d"
format_stamp: "Formatted at 2026-09-29 19:10:34 on dist-test-slave-wftq"
server_key: "0e5e55dd13270c1193f4575a3a235ad0"
server_key_iv: "a0910f1f8b7a43712480cf5d52a1545a"
server_key_version: "encryptionkey@0"
I20260929 19:10:34.260448 16715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:34.270512 16715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:34.272516 16715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:34.272640 16715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:34.276973 16715 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.61:43681
I20260929 19:10:34.277051 16782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.61:43681 every 8 connection(s)
I20260929 19:10:34.277415 16715 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/data/info.pb
I20260929 19:10:34.278457 16783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:34.280016 16783 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709034, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.61@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM
I20260929 19:10:34.287034 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 16715
I20260929 19:10:34.290136 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.280737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54574 (local address 127.9.236.62:38041)
0929 19:10:34.280858 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.280861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.281123 (+ 262us) server_negotiation.cc:407] Connection header received
0929 19:10:34.281742 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.281744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.281758 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.281839 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.282613 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.283259 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.284331 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.284531 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.284586 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.286898 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.286899 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.286900 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.286907 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.288469 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.288961 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.288962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.288962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.289016 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.289371 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.289371 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.289371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.289534 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.289591 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.289961 (+ 370us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.290034 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 19:10:34.290324 16785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.280669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.0.0.1:54574)
0929 19:10:34.280934 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.280980 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.281636 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.281915 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.281920 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.282222 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:34.282519 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.282526 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.283341 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.283342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.284256 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.284258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.284598 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.284599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.284726 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.285318 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:34.285327 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:34.286804 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:34.288557 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.288561 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.288562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.288875 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.289081 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.289082 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.289083 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.289247 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.289632 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:34.289636 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:34.289814 (+ 178us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.289936 (+ 122us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.290112 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":138,"threads_started":1}
I20260929 19:10:34.291038 16783 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.236.61@KRBTEST.COM
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709034, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.61@KRBTEST.COM for oryx/127.9.236.61@KRBTEST.COM
I20260929 19:10:34.296450 16787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.287581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59472 (local address 127.9.236.61:43681)
0929 19:10:34.287745 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.287751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.287974 (+ 223us) server_negotiation.cc:407] Connection header received
0929 19:10:34.288050 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.288052 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.288065 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.288136 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.288827 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.289952 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.290895 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.291434 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.291510 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.293738 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.293746 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.293746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.293752 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.295295 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.295645 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.295650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.295651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.295718 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.295878 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.295880 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.295880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.296039 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.296214 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.296257 (+ 43us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.296340 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260929 19:10:34.298447 16785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.291541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:43681 (local address 127.0.0.1:59474)
0929 19:10:34.291764 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.291776 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.291826 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.291938 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.291939 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.292086 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:34.292128 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.292129 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.292752 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.292753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.293258 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.293260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.293632 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.293633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.293693 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.294049 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:34.294053 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:34.295395 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:34.297529 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.297529 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.297530 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.297693 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.297847 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.297848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.297848 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.297891 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.298098 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:34.298099 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:34.298141 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.298200 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.298310 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206}
I20260929 19:10:34.298467 16788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.291632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59474 (local address 127.9.236.61:43681)
0929 19:10:34.291779 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.291780 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.291811 (+ 31us) server_negotiation.cc:407] Connection header received
0929 19:10:34.291873 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.291874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.291883 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.291938 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.292187 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.292669 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.293330 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.293575 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.293627 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.295484 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.295484 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.295485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.295490 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.297458 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.297752 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.297752 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.297753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.297800 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.297929 (+ 129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.297930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.297930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.298028 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.298085 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.298226 (+ 141us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.298358 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
I20260929 19:10:34.300077 16783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: Bootstrap starting.
I20260929 19:10:34.300741 16783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:34.301011 16783 log.cc:824] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:34.301841 16783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: No bootstrap required, opened a new log
I20260929 19:10:34.303411 16783 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } }
I20260929 19:10:34.303543 16783 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:34.303570 16783 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0c0560533814a7cbebc5c5940f38a3d, State: Initialized, Role: FOLLOWER
I20260929 19:10:34.303691 16783 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } }
I20260929 19:10:34.304015 16783 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:34.303982 16790 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } } }
I20260929 19:10:34.304092 16790 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [sys.catalog]: This master's current role is: FOLLOWER
W20260929 19:10:34.305792 16801 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:10:34.305964 16801 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:10:34.307533 16715 master_runner.cc:428] Detected that this master f0c0560533814a7cbebc5c5940f38a3d is joining an existing cluster
I20260929 19:10:34.307622 16715 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.236.61:43681
I20260929 19:10:34.315501 16808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.308705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:43681 (local address 127.0.0.1:59478)
0929 19:10:34.309054 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.309063 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.309150 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.309310 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.309311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.309569 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:34.309647 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.309649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.310533 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.310534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.311491 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.311493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.311902 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.311903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.311971 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.312369 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:34.312373 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:34.312744 (+ 371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:34.314247 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.314248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.314249 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.314510 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.314681 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.314682 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.314682 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.314753 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.315045 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:34.315046 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:34.315096 (+ 50us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.315186 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.315315 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":19,"thread_start_us":113,"threads_started":1}
I20260929 19:10:34.315570 16788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.308909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59478 (local address 127.9.236.61:43681)
0929 19:10:34.308955 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.308957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.309133 (+ 176us) server_negotiation.cc:407] Connection header received
0929 19:10:34.309203 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.309204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.309215 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.309304 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.309710 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.310451 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.311587 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.311821 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.311886 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.312822 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.312824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.312824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.312830 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.314178 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.314565 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.314566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.314567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.314620 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.314821 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.314822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.314823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.314973 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.315045 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.315224 (+ 179us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.315385 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":28}
I20260929 19:10:34.317227 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.308766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54586 (local address 127.9.236.62:38041)
0929 19:10:34.308830 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.308832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.308980 (+ 148us) server_negotiation.cc:407] Connection header received
0929 19:10:34.309085 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.309086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.309140 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.309171 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.309776 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.310625 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.312603 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.312923 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.312966 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.314630 (+ 1664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.314631 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.314631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.314639 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.316035 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.316406 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.316407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.316408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.316479 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.316665 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.316666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.316666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.316808 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.316885 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.317016 (+ 131us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.317131 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 19:10:34.317214 16807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.308705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.0.0.1:54586)
0929 19:10:34.308898 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.308910 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.309011 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.309242 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.309243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.309541 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:34.309655 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.309657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.311616 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.311617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.312478 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.312480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.313053 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.313054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.313143 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.313551 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:34.313564 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:34.314544 (+ 980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:34.316106 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.316108 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.316109 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.316328 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.316551 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.316552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.316552 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.316605 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.316873 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:34.316873 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:34.316918 (+ 45us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.316970 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.317056 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":79,"threads_started":1}
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709033, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.62@KRBTEST.COM for oryx/127.9.236.61@KRBTEST.COM
I20260929 19:10:34.326160 16788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.319577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59486 (local address 127.9.236.61:43681)
0929 19:10:34.319618 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.319619 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.319791 (+ 172us) server_negotiation.cc:407] Connection header received
0929 19:10:34.319899 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.319899 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.319909 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.319955 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.320399 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.320859 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.321665 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.321865 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.321918 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.323948 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.323949 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.323950 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.323956 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.325146 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.325489 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.325490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.325491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.325540 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.325692 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.325692 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.325693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.325796 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.325846 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.326003 (+ 157us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.326072 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:10:34.326171 16809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.319529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:43681 (local address 127.0.0.1:59486)
0929 19:10:34.319687 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.319731 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.319804 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.319969 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.319975 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.320277 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:34.320338 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.320340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.320924 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.320925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.321590 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.321593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.321922 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.321923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.321996 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.322364 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:34.322369 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:34.323873 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:34.325237 (+ 1364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.325244 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.325247 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.325425 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.325589 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.325590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.325590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.325645 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.325846 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:34.325853 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:34.325903 (+ 50us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.325973 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.326076 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260929 19:10:34.327044 16251 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.236.61:43681: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d"
member_type: NON_VOTER
last_known_addr {
host: "127.9.236.61"
port: 43681
}
attrs {
promote: true
}
}
dest_uuid: "64eaa0faf742494680dadbf96bb413f0"
cas_config_opid_index: -1
I20260929 19:10:34.327395 16251 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: NON_VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: true } }
I20260929 19:10:34.327913 16810 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f0c0560533814a7cbebc5c5940f38a3d (127.9.236.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: NON_VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: true } } }
I20260929 19:10:34.328253 16757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:34.328814 16810 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.236.61:43681
I20260929 19:10:34.328902 16810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "64eaa0faf742494680dadbf96bb413f0" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: NON_VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: true } } }
I20260929 19:10:34.328963 16810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:34.329401 16757 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 1 FOLLOWER]: Refusing update from remote peer 64eaa0faf742494680dadbf96bb413f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260929 19:10:34.329789 16715 master.cc:537] Master@127.9.236.61:43681 shutting down...
I20260929 19:10:34.329774 16810 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: NON_VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 19:10:34.330538 16235 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260929 19:10:34.331067 16235 consensus_peers.cc:487] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 -> Peer f0c0560533814a7cbebc5c5940f38a3d (127.9.236.61:43681): Couldn't send request to peer f0c0560533814a7cbebc5c5940f38a3d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 19:10:34.332908 16715 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:34.332996 16715 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:34.333048 16715 tablet_replica.cc:330] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: stopping tablet replica
I20260929 19:10:34.345639 16715 master.cc:552] Master@127.9.236.61:43681 shutdown complete.
I20260929 19:10:34.345685 16715 master_runner.cc:305] Clearing existing system tablet
I20260929 19:10:34.345790 16715 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:34.345837 16715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:34.346418 16715 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:10:34.346884 16815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:34.347157 16715 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:34.347222 16715 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 19:10:34.347247 16715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/wal
uuid: "f0c0560533814a7cbebc5c5940f38a3d"
format_stamp: "Formatted at 2026-09-29 19:10:34 on dist-test-slave-wftq"
server_key: "0e5e55dd13270c1193f4575a3a235ad0"
server_key_iv: "a0910f1f8b7a43712480cf5d52a1545a"
server_key_version: "encryptionkey@0"
I20260929 19:10:34.348017 16715 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:34.349467 16715 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 19:10:34.349530 16715 log.cc:1196] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260929 19:10:34.349681 16715 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: Deleting consensus metadata
I20260929 19:10:34.349799 16715 master_runner.cc:315] Copying system tablet from 127.9.236.62:38041
I20260929 19:10:34.350345 16715 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: tablet copy: Beginning tablet copy session from remote peer at address 127.9.236.62:38041
I20260929 19:10:34.356698 16821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.350603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.0.0.1:54600)
0929 19:10:34.350784 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.350792 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.350848 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.351016 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.351017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.351167 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:34.351227 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.351229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.352219 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.352220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.352817 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.352818 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.353334 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.353335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.353387 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.353813 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:34.353817 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:34.354087 (+ 270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:34.355771 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.355772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.355772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.355938 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.356113 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.356113 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.356114 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.356156 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.356429 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:34.356429 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:34.356473 (+ 44us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.356537 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.356617 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":58,"threads_started":1}
I20260929 19:10:34.356767 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.350705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54600 (local address 127.9.236.62:38041)
0929 19:10:34.350777 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.350779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.350852 (+ 73us) server_negotiation.cc:407] Connection header received
0929 19:10:34.350930 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.350931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.350946 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.351009 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.351295 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.352133 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.352893 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.353245 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.353316 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.354172 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.354174 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.354174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.354181 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.355688 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.356005 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.356006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.356006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.356062 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.356208 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.356208 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.356209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.356363 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.356429 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.356582 (+ 153us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.356660 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 19:10:34.357226 16271 tablet_copy_service.cc:140] P 64eaa0faf742494680dadbf96bb413f0: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f0c0560533814a7cbebc5c5940f38a3d ({username='oryx', principal='oryx/127.9.236.61@KRBTEST.COM'} at 127.0.0.1:54600)
I20260929 19:10:34.357322 16271 tablet_copy_service.cc:161] P 64eaa0faf742494680dadbf96bb413f0: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f0c0560533814a7cbebc5c5940f38a3d at {username='oryx', principal='oryx/127.9.236.61@KRBTEST.COM'} at 127.0.0.1:54600: session id = f0c0560533814a7cbebc5c5940f38a3d-00000000000000000000000000000000
I20260929 19:10:34.357904 16271 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 19:10:34.358470 16715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 19:10:34.359300 16715 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: tablet copy: Starting download of 0 data blocks...
I20260929 19:10:34.359408 16715 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: tablet copy: Starting download of 1 WAL segments...
I20260929 19:10:34.360138 16715 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 19:10:34.361361 16271 tablet_copy_service.cc:342] P 64eaa0faf742494680dadbf96bb413f0: Request end of tablet copy session f0c0560533814a7cbebc5c5940f38a3d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.236.61@KRBTEST.COM'} at 127.0.0.1:54600
I20260929 19:10:34.361409 16271 tablet_copy_service.cc:433] P 64eaa0faf742494680dadbf96bb413f0: ending tablet copy session f0c0560533814a7cbebc5c5940f38a3d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f0c0560533814a7cbebc5c5940f38a3d
I20260929 19:10:34.364022 16715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:34.365257 16825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:34.365345 16822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:34.365371 16715 server_base.cc:1034] running on GCE node
W20260929 19:10:34.365543 16823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:34.365733 16715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:34.365885 16715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:34.367010 16715 hybrid_clock.cc:648] HybridClock initialized: now 1790709034366995 us; error 31 us; skew 500 ppm
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709034, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:34.368440 16715 init.cc:377] Logged in from keytab as oryx/127.9.236.61@KRBTEST.COM (short username oryx)
I20260929 19:10:34.368677 16715 webserver.cc:460] Webserver started at http://127.9.236.61:40611/ using document root <none> and password file <none>
I20260929 19:10:34.368794 16715 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:34.368829 16715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:34.369403 16715 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:10:34.369875 16832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:34.369977 16715 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:34.370023 16715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/wal
uuid: "f0c0560533814a7cbebc5c5940f38a3d"
format_stamp: "Formatted at 2026-09-29 19:10:34 on dist-test-slave-wftq"
server_key: "0e5e55dd13270c1193f4575a3a235ad0"
server_key_iv: "a0910f1f8b7a43712480cf5d52a1545a"
server_key_version: "encryptionkey@0"
I20260929 19:10:34.370082 16715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:34.375828 16715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:34.377272 16715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:34.380007 16715 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.61:43681
I20260929 19:10:34.380426 16715 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-1/data/info.pb
I20260929 19:10:34.380445 16884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.61:43681 every 8 connection(s)
I20260929 19:10:34.381394 16885 sys_catalog.cc:263] Verifying existing consensus state
I20260929 19:10:34.381950 16885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: Bootstrap starting.
I20260929 19:10:34.385071 16885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:10:34.385366 16885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: Bootstrap complete.
I20260929 19:10:34.385622 16885 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: NON_VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: true } }
I20260929 19:10:34.385722 16885 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f0c0560533814a7cbebc5c5940f38a3d, State: Initialized, Role: LEARNER
I20260929 19:10:34.385771 16885 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: NON_VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: true } }
I20260929 19:10:34.385990 16885 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:34.385987 16888 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: NON_VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: true } } }
I20260929 19:10:34.386138 16888 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [sys.catalog]: This master's current role is: LEARNER
I20260929 19:10:34.387741 16899 catalog_manager.cc:1259] Loaded cluster ID: 5efb130dadec41a89dcc31b7161887e6
I20260929 19:10:34.387792 16899 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: loading cluster ID for follower catalog manager: success
I20260929 19:10:34.388859 16899 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: acquiring CA information for follower catalog manager: success
I20260929 19:10:34.389160 16899 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 19:10:34.403982 16900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.398213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59494 (local address 127.9.236.61:43681)
0929 19:10:34.398374 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.398376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.398383 (+ 7us) server_negotiation.cc:407] Connection header received
0929 19:10:34.398403 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.398404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.398414 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.398469 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.399001 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.399685 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.400350 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.400540 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.400603 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.401661 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.401662 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.401663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.401669 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.402868 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.403150 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.403151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.403152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.403202 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.403357 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.403357 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.403358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.403455 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.403541 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.403643 (+ 102us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.403728 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":104,"threads_started":1}
WARNING: no policy specified for oryx/127.9.236.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.236.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.236.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.60.keytab.
Entry for principal oryx/127.9.236.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.60.keytab.
I20260929 19:10:34.435444 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.60:41353
--webserver_interface=127.9.236.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.60.keytab
--principal=oryx/127.9.236.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:38041,127.9.236.61:43681,127.9.236.60:41353
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:34.528861 16903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:34.529130 16903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:34.529155 16903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:34.530315 16903 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:34.530355 16903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:34.530368 16903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:34.530380 16903 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:34.530390 16903 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:34.531942 16903 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:38041,127.9.236.61:43681,127.9.236.60:41353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.60.keytab
--principal=oryx/127.9.236.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.60:41353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.236.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:34.532178 16903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:34.532390 16903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:34.537737 16909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:34.537737 16911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:34.537791 16908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:34.538085 16903 server_base.cc:1034] running on GCE node
I20260929 19:10:34.538357 16903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:34.538684 16903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:34.539855 16903 hybrid_clock.cc:648] HybridClock initialized: now 1790709034539830 us; error 42 us; skew 500 ppm
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709034, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:34.542510 16903 init.cc:377] Logged in from keytab as oryx/127.9.236.60@KRBTEST.COM (short username oryx)
I20260929 19:10:34.542994 16903 webserver.cc:460] Webserver started at http://127.9.236.60:45081/ using document root <none> and password file <none>
I20260929 19:10:34.543224 16903 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:34.543272 16903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:34.543392 16903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:34.544495 16903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/data/instance:
uuid: "fde63ef43d8a440e96811f6d118232f2"
format_stamp: "Formatted at 2026-09-29 19:10:34 on dist-test-slave-wftq"
server_key: "fb5072310b9ce18615c8363659cf2f18"
server_key_iv: "532016fbecd085e9ca7e42910a2d2663"
server_key_version: "encryptionkey@0"
I20260929 19:10:34.544832 16903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/wal/instance:
uuid: "fde63ef43d8a440e96811f6d118232f2"
format_stamp: "Formatted at 2026-09-29 19:10:34 on dist-test-slave-wftq"
server_key: "fb5072310b9ce18615c8363659cf2f18"
server_key_iv: "532016fbecd085e9ca7e42910a2d2663"
server_key_version: "encryptionkey@0"
I20260929 19:10:34.546197 16903 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:10:34.547010 16918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:34.547240 16903 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:34.547330 16903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/wal
uuid: "fde63ef43d8a440e96811f6d118232f2"
format_stamp: "Formatted at 2026-09-29 19:10:34 on dist-test-slave-wftq"
server_key: "fb5072310b9ce18615c8363659cf2f18"
server_key_iv: "532016fbecd085e9ca7e42910a2d2663"
server_key_version: "encryptionkey@0"
I20260929 19:10:34.547402 16903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:34.567065 16903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:34.569550 16903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:34.569703 16903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:34.574486 16903 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.60:41353
I20260929 19:10:34.574527 16970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.60:41353 every 8 connection(s)
I20260929 19:10:34.574940 16903 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/data/info.pb
I20260929 19:10:34.575999 16971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:34.577605 16971 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:10:34.581370 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 16903
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709034, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.60@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.236.60@KRBTEST.COM
I20260929 19:10:34.587658 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.578458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54606 (local address 127.9.236.62:38041)
0929 19:10:34.578542 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.578544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.578645 (+ 101us) server_negotiation.cc:407] Connection header received
0929 19:10:34.579212 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.579213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.579237 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.579287 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.579976 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.580484 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.581560 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.581772 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.581842 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.584518 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.584520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.584521 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.584529 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.586045 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.586590 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.586591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.586592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.586650 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.586916 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.586917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.586918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.587096 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.587184 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.587463 (+ 279us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.587555 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 19:10:34.587836 16973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.578295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.0.0.1:54606)
0929 19:10:34.578508 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.578556 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.579109 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.579365 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.579369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.579655 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:34.579898 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.579902 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.580562 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.580563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.581451 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.581454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.581891 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.581894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.582104 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.582682 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:34.582691 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:34.584393 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:34.586133 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.586140 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.586142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.586443 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.586720 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.586721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.586721 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.586817 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.587173 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:34.587177 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:34.587310 (+ 133us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.587411 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.587529 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":88,"threads_started":1}
I20260929 19:10:34.588579 16971 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:10:34.590883 16974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.581943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57048 (local address 127.9.236.60:41353)
0929 19:10:34.582150 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.582154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.582174 (+ 20us) server_negotiation.cc:407] Connection header received
0929 19:10:34.582199 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.582201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.582223 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.582349 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.582970 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.583831 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.584683 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.584976 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.585031 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.587995 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.588002 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.588002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.588008 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.589784 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.590100 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.590103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.590103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.590168 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.590389 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.590389 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.590390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.590541 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.590667 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.590746 (+ 79us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.590812 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":146,"threads_started":1}
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709034, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.60@KRBTEST.COM for oryx/127.9.236.61@KRBTEST.COM
I20260929 19:10:34.595175 16973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.588851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:43681 (local address 127.0.0.1:59498)
0929 19:10:34.588964 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.588978 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.589017 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.589209 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.589210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.589384 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:34.589430 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.589431 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.590305 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.590306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.591255 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.591257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.591722 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.591722 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.591794 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.592122 (+ 328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:34.592127 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:34.593242 (+ 1115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:34.594441 (+ 1199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.594442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.594442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.594585 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.594747 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.594748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.594748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.594792 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.594965 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:34.594965 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:34.594997 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.595047 (+ 50us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.595111 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260929 19:10:34.595242 16900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.588930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59498 (local address 127.9.236.61:43681)
0929 19:10:34.589011 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.589013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.589024 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:10:34.589084 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.589086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.589148 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.589206 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.589513 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.590219 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.591333 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.591660 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.591716 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.593319 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.593320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.593321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.593326 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.594366 (+ 1040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.594648 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.594648 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.594649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.594693 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.594833 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.594834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.594834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.594921 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.594968 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.595088 (+ 120us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.595167 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 19:10:34.595710 16971 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709034, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.60@KRBTEST.COM for oryx/127.9.236.60@KRBTEST.COM
I20260929 19:10:34.601640 16973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.595965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:41353 (local address 127.0.0.1:57056)
0929 19:10:34.596052 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.596103 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.596153 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.596265 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.596266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.596391 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:34.596423 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.596424 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.597019 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.597020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.597536 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.597537 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.597852 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.597853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.597898 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.598278 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:34.598282 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:34.599369 (+ 1087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:34.600609 (+ 1240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.600609 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.600610 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.600766 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.601029 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.601030 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.601030 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.601092 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.601392 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:34.601393 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:34.601428 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.601488 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.601574 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260929 19:10:34.601730 16974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.596012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57056 (local address 127.9.236.60:41353)
0929 19:10:34.596049 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.596051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.596148 (+ 97us) server_negotiation.cc:407] Connection header received
0929 19:10:34.596203 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.596204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.596215 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.596259 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.596483 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.596956 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.597599 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.597800 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.597853 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.599445 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.599447 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.599447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.599453 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.600539 (+ 1086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.600867 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.600868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.600868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.600926 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.601227 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.601228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.601228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.601339 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.601401 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.601517 (+ 116us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.601629 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":16}
I20260929 19:10:34.603253 16971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2: Bootstrap starting.
I20260929 19:10:34.604162 16971 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:34.604413 16971 log.cc:824] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:34.605288 16971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2: No bootstrap required, opened a new log
I20260929 19:10:34.606909 16971 raft_consensus.cc:348] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } }
I20260929 19:10:34.607054 16971 raft_consensus.cc:374] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:34.607079 16971 raft_consensus.cc:729] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fde63ef43d8a440e96811f6d118232f2, State: Initialized, Role: FOLLOWER
I20260929 19:10:34.607180 16971 consensus_queue.cc:260] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } }
I20260929 19:10:34.607465 16971 sys_catalog.cc:564] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:34.607448 16976 sys_catalog.cc:455] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } } }
I20260929 19:10:34.607574 16976 sys_catalog.cc:458] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 19:10:34.609225 16987 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:10:34.609283 16987 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:10:34.612089 16903 master_runner.cc:428] Detected that this master fde63ef43d8a440e96811f6d118232f2 is joining an existing cluster
I20260929 19:10:34.612133 16903 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.236.60:41353
I20260929 19:10:34.619688 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.613090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.0.0.1:54608)
0929 19:10:34.613272 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.613283 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.613365 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.613546 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.613547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.613743 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:34.613816 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.613819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.614742 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.614742 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.615647 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.615649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.616135 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.616135 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.616192 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.616837 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:34.616842 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:34.617253 (+ 411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:34.618758 (+ 1505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.618759 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.618759 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.618927 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.619129 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.619131 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.619131 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.619187 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.619395 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:34.619395 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:34.619437 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.619508 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.619607 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":130,"threads_started":1}
I20260929 19:10:34.619942 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.613151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54608 (local address 127.9.236.62:38041)
0929 19:10:34.613200 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.613202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.613337 (+ 135us) server_negotiation.cc:407] Connection header received
0929 19:10:34.613434 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.613435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.613447 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.613477 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.613885 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.614660 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.615727 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.616018 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.616070 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.617333 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.617335 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.617335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.617342 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.618689 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.619025 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.619026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.619027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.619078 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.619235 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.619235 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.619236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.619345 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.619399 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.619802 (+ 403us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.619881 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709033, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.62@KRBTEST.COM for oryx/127.9.236.60@KRBTEST.COM
I20260929 19:10:34.636868 16809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.622328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:41353 (local address 127.0.0.1:57074)
0929 19:10:34.622393 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.622406 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.622457 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.622688 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.622689 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.622880 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:34.622931 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.622933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.623648 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.623649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.624472 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.624475 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.624858 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.624859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.624923 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.625411 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:34.625416 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:34.627002 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:34.635631 (+ 8629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.635632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.635633 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.635881 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.636099 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.636100 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.636100 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.636164 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.636493 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:34.636494 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:34.636559 (+ 65us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.636609 (+ 50us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.636741 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 19:10:34.637337 16996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.622382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57074 (local address 127.9.236.60:41353)
0929 19:10:34.622555 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.622556 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.622564 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:10:34.622596 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.622596 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.622605 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.622663 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.623012 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.623570 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.624567 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.624786 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.624847 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.627127 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.627129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.627129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.627136 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.635534 (+ 8398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.635965 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.635966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.635967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.636025 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.636229 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.636230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.636230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.636380 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.637072 (+ 692us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.637137 (+ 65us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.637218 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":114,"threads_started":1}
I20260929 19:10:34.637514 16974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.613400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57058 (local address 127.9.236.60:41353)
0929 19:10:34.613436 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.613438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.613987 (+ 549us) server_negotiation.cc:407] Connection header received
0929 19:10:34.614054 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.614054 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.614062 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.614084 (+ 22us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.614447 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.615113 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.616079 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.616315 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.616352 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.634434 (+ 18082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.634436 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.634436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.634442 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.636228 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.636635 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.636636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.636636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.636690 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.636928 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.636928 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.636929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.637061 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.637133 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.637323 (+ 190us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.637412 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:10:34.637857 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.613483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:41353 (local address 127.0.0.1:57058)
0929 19:10:34.613926 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.613935 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.614005 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.614129 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.614130 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.614325 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:34.614382 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.614383 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.615191 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.615191 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.616003 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.616005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.616412 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.616412 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.616470 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.633707 (+ 17237us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:34.633712 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:34.634319 (+ 607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:34.636297 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.636299 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.636299 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.636547 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.636776 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.636777 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.636777 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.636869 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.637172 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:34.637172 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:34.637225 (+ 53us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.637655 (+ 430us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.637762 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":71,"threads_started":1}
I20260929 19:10:34.638057 16251 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.236.60:41353: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "fde63ef43d8a440e96811f6d118232f2"
member_type: NON_VOTER
last_known_addr {
host: "127.9.236.60"
port: 41353
}
attrs {
promote: true
}
}
dest_uuid: "64eaa0faf742494680dadbf96bb413f0"
cas_config_opid_index: 5
I20260929 19:10:34.638347 16251 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: NON_VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: true } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: NON_VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: true } }
I20260929 19:10:34.638901 16811 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER fde63ef43d8a440e96811f6d118232f2 (127.9.236.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: NON_VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: true } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: NON_VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: true } } }
I20260929 19:10:34.639227 16900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.613350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59510 (local address 127.9.236.61:43681)
0929 19:10:34.613396 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.613397 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.613519 (+ 122us) server_negotiation.cc:407] Connection header received
0929 19:10:34.613636 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.613637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.613650 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.613708 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.614119 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.614884 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.615983 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.616264 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.616306 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.634811 (+ 18505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.634813 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.634814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.634825 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.636905 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.638372 (+ 1467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.638373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.638374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.638431 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.638642 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.638644 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.638644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.638795 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.638843 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.639043 (+ 200us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.639141 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 19:10:34.639575 16994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.613159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:43681 (local address 127.0.0.1:59510)
0929 19:10:34.613458 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.613469 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.613544 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.613732 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.613733 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.613942 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:34.614018 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.614020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.614983 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.614983 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.615886 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.615888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.616383 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.616384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.616448 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.634098 (+ 17650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:34.634104 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:34.634682 (+ 578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:34.637339 (+ 2657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.637340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.637340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.638302 (+ 962us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.638508 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.638508 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.638509 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.638575 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.638892 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:34.638893 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:34.638933 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.639327 (+ 394us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.639470 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":89,"threads_started":1}
I20260929 19:10:34.639892 16811 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.236.60:41353
I20260929 19:10:34.640017 16810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "64eaa0faf742494680dadbf96bb413f0" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: NON_VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: true } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: NON_VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: true } } }
I20260929 19:10:34.640120 16810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:34.640650 16945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:34.641675 16945 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 1 FOLLOWER]: Refusing update from remote peer 64eaa0faf742494680dadbf96bb413f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 19:10:34.642154 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 16313
I20260929 19:10:34.642233 16903 master.cc:537] Master@127.9.236.60:41353 shutting down...
I20260929 19:10:34.645664 16810 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: NON_VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 19:10:34.646479 16238 consensus_peers.cc:487] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 -> Peer fde63ef43d8a440e96811f6d118232f2 (127.9.236.60:41353): Couldn't send request to peer fde63ef43d8a440e96811f6d118232f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 19:10:34.646486 16809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.638801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:43681 (local address 127.0.0.1:51674)
0929 19:10:34.639466 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.639478 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.639528 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.639703 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.639705 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.639707 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:34.639764 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.639766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.640547 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.640548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.641472 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.641474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.646148 (+ 4674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.646149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.646250 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.646288 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.646351 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.646407 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646}
I20260929 19:10:34.646636 16900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.638902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51674 (local address 127.9.236.61:43681)
0929 19:10:34.639308 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.639310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.639578 (+ 268us) server_negotiation.cc:407] Connection header received
0929 19:10:34.639598 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.639599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.639627 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.639688 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:34.639832 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.640470 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.641544 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.642059 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.642117 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.642199 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.646517 (+ 4318us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.646568 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":86,"threads_started":1}
I20260929 19:10:34.648880 16859 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 1 LEARNER]: Refusing update from remote peer 64eaa0faf742494680dadbf96bb413f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 19:10:34.649148 16810 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: NON_VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260929 19:10:34.649199 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:46669
--local_ip_for_outbound_sockets=127.9.236.1
--tserver_master_addrs=127.9.236.62:38041
--webserver_port=34031
--webserver_interface=127.9.236.1
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.1.keytab
--principal=oryx/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.236.62:38041,127.9.236.61:43681
--tserver_master_addrs=127.9.236.62:38041,127.9.236.61:43681,127.9.236.60:41353 with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:10:34.649751 16859 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER fde63ef43d8a440e96811f6d118232f2 (127.9.236.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: NON_VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: true } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: NON_VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: true } } }
I20260929 19:10:34.651046 17000 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "64eaa0faf742494680dadbf96bb413f0" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: NON_VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: true } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: NON_VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: true } } }
I20260929 19:10:34.651167 16888 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64eaa0faf742494680dadbf96bb413f0. Latest consensus state: current_term: 1 leader_uuid: "64eaa0faf742494680dadbf96bb413f0" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: NON_VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: true } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: NON_VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: true } } }
I20260929 19:10:34.651298 16888 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [sys.catalog]: This master's current role is: LEARNER
I20260929 19:10:34.651180 17000 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [sys.catalog]: This master's current role is: LEARNER
I20260929 19:10:34.652056 16810 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0: attempting to promote NON_VOTER f0c0560533814a7cbebc5c5940f38a3d to VOTER
I20260929 19:10:34.652278 16810 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: NON_VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: true } }
I20260929 19:10:34.652326 16903 raft_consensus.cc:2229] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:34.652433 16903 raft_consensus.cc:2258] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:34.652508 16903 tablet_replica.cc:330] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2: stopping tablet replica
I20260929 19:10:34.652801 16859 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 1 LEARNER]: Refusing update from remote peer 64eaa0faf742494680dadbf96bb413f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260929 19:10:34.652974 16811 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260929 19:10:34.653178 16238 consensus_peers.cc:487] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 -> Peer fde63ef43d8a440e96811f6d118232f2 (127.9.236.60:41353): Couldn't send request to peer fde63ef43d8a440e96811f6d118232f2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 19:10:34.654039 16811 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f0c0560533814a7cbebc5c5940f38a3d (127.9.236.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: NON_VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: true } } }
I20260929 19:10:34.655166 16811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "64eaa0faf742494680dadbf96bb413f0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: NON_VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: true } } }
I20260929 19:10:34.655246 16811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:34.655294 16811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "64eaa0faf742494680dadbf96bb413f0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: NON_VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: true } } }
I20260929 19:10:34.655329 16811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:34.655580 16859 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f0c0560533814a7cbebc5c5940f38a3d (127.9.236.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: NON_VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: true } } }
I20260929 19:10:34.656528 17000 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "64eaa0faf742494680dadbf96bb413f0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: NON_VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: true } } }
I20260929 19:10:34.656606 17000 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:10:34.666338 16903 master.cc:552] Master@127.9.236.60:41353 shutdown complete.
I20260929 19:10:34.666402 16903 master_runner.cc:305] Clearing existing system tablet
I20260929 19:10:34.666530 16903 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:34.666586 16903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:34.667412 16903 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:34.668028 17003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:34.668179 16903 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:34.668236 16903 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:34.668262 16903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/wal
uuid: "fde63ef43d8a440e96811f6d118232f2"
format_stamp: "Formatted at 2026-09-29 19:10:34 on dist-test-slave-wftq"
server_key: "fb5072310b9ce18615c8363659cf2f18"
server_key_iv: "532016fbecd085e9ca7e42910a2d2663"
server_key_version: "encryptionkey@0"
I20260929 19:10:34.669346 16903 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:34.671083 16903 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 19:10:34.671172 16903 log.cc:1196] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260929 19:10:34.671406 16903 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2: Deleting consensus metadata
I20260929 19:10:34.671545 16903 master_runner.cc:315] Copying system tablet from 127.9.236.62:38041
I20260929 19:10:34.672303 16903 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2: tablet copy: Beginning tablet copy session from remote peer at address 127.9.236.62:38041
I20260929 19:10:34.679818 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.672766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34404 (local address 127.9.236.62:38041)
0929 19:10:34.672830 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.672832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.673267 (+ 435us) server_negotiation.cc:407] Connection header received
0929 19:10:34.673375 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.673377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.673390 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.673420 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.673860 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.674637 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.675641 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.675950 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.675986 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.677214 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.677215 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.677215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.677222 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.678726 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.679076 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.679077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.679078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.679125 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.679293 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.679294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.679294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.679415 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.679457 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.679641 (+ 184us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.679724 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 19:10:34.680341 17009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.672667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.0.0.1:34404)
0929 19:10:34.673185 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.673197 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.673302 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.673487 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.673487 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.673721 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:34.673775 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.673777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.674739 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.674740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.675548 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.675551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.676061 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.676062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.676111 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.676714 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:34.676718 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:34.677088 (+ 370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:34.678810 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.678811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.678812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.679004 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.679184 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.679185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.679185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.679234 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.679499 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:34.679500 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:34.679541 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.680125 (+ 584us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.680232 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":84,"threads_started":1}
I20260929 19:10:34.680819 16271 tablet_copy_service.cc:140] P 64eaa0faf742494680dadbf96bb413f0: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer fde63ef43d8a440e96811f6d118232f2 ({username='oryx', principal='oryx/127.9.236.60@KRBTEST.COM'} at 127.0.0.1:34404)
I20260929 19:10:34.680881 16271 tablet_copy_service.cc:161] P 64eaa0faf742494680dadbf96bb413f0: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer fde63ef43d8a440e96811f6d118232f2 at {username='oryx', principal='oryx/127.9.236.60@KRBTEST.COM'} at 127.0.0.1:34404: session id = fde63ef43d8a440e96811f6d118232f2-00000000000000000000000000000000
I20260929 19:10:34.681562 16271 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 19:10:34.682099 16903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 19:10:34.683480 16903 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2: tablet copy: Starting download of 0 data blocks...
I20260929 19:10:34.683629 16903 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2: tablet copy: Starting download of 1 WAL segments...
I20260929 19:10:34.684373 16903 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 19:10:34.685361 16271 tablet_copy_service.cc:342] P 64eaa0faf742494680dadbf96bb413f0: Request end of tablet copy session fde63ef43d8a440e96811f6d118232f2-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.236.60@KRBTEST.COM'} at 127.0.0.1:34404
I20260929 19:10:34.685424 16271 tablet_copy_service.cc:433] P 64eaa0faf742494680dadbf96bb413f0: ending tablet copy session fde63ef43d8a440e96811f6d118232f2-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer fde63ef43d8a440e96811f6d118232f2
I20260929 19:10:34.688905 16903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:34.690377 17013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:34.690352 17010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:34.690814 16903 server_base.cc:1034] running on GCE node
W20260929 19:10:34.690632 17011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:34.691066 16903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:34.691242 16903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:34.692353 16903 hybrid_clock.cc:648] HybridClock initialized: now 1790709034692346 us; error 30 us; skew 500 ppm
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709034, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:34.694010 16903 init.cc:377] Logged in from keytab as oryx/127.9.236.60@KRBTEST.COM (short username oryx)
I20260929 19:10:34.694308 16903 webserver.cc:460] Webserver started at http://127.9.236.60:44531/ using document root <none> and password file <none>
I20260929 19:10:34.694416 16903 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:34.694443 16903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:34.695222 16903 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:34.695842 17020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:34.696002 16903 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:34.696055 16903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/wal
uuid: "fde63ef43d8a440e96811f6d118232f2"
format_stamp: "Formatted at 2026-09-29 19:10:34 on dist-test-slave-wftq"
server_key: "fb5072310b9ce18615c8363659cf2f18"
server_key_iv: "532016fbecd085e9ca7e42910a2d2663"
server_key_version: "encryptionkey@0"
I20260929 19:10:34.696110 16903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:34.714659 16903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:34.717408 16903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:34.720860 16903 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.60:41353
I20260929 19:10:34.721238 16903 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/master-2/data/info.pb
I20260929 19:10:34.722275 17073 sys_catalog.cc:263] Verifying existing consensus state
I20260929 19:10:34.722813 17073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2: Bootstrap starting.
I20260929 19:10:34.725224 17072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.60:41353 every 8 connection(s)
I20260929 19:10:34.725914 17073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:10:34.726115 17073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2: Bootstrap complete.
I20260929 19:10:34.726406 17073 raft_consensus.cc:348] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: NON_VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: true } }
I20260929 19:10:34.726504 17073 raft_consensus.cc:729] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: fde63ef43d8a440e96811f6d118232f2, State: Initialized, Role: LEARNER
I20260929 19:10:34.726538 17073 consensus_queue.cc:260] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: NON_VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: true } }
I20260929 19:10:34.726727 17073 sys_catalog.cc:564] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:34.726734 17076 sys_catalog.cc:455] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: NON_VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: true } } }
I20260929 19:10:34.727025 17076 sys_catalog.cc:458] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [sys.catalog]: This master's current role is: LEARNER
I20260929 19:10:34.728302 17087 catalog_manager.cc:1259] Loaded cluster ID: 5efb130dadec41a89dcc31b7161887e6
I20260929 19:10:34.728398 17087 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2: loading cluster ID for follower catalog manager: success
I20260929 19:10:34.729693 17087 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2: acquiring CA information for follower catalog manager: success
I20260929 19:10:34.730010 17087 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 19:10:34.756428 16999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:34.756656 16999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:34.756685 16999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:34.757906 16999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:34.757952 16999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:34.757978 16999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:10:34.759469 16999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.1.keytab
--principal=oryx/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:46669
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=34031
--tserver_master_addrs=127.9.236.62:38041,127.9.236.61:43681,127.9.236.60:41353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:34.759704 16999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:34.759976 16999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:34.765129 17095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:34.765196 17093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:34.765201 17092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:34.765508 16999 server_base.cc:1034] running on GCE node
I20260929 19:10:34.765749 16999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:34.766021 16999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:34.767174 16999 hybrid_clock.cc:648] HybridClock initialized: now 1790709034767134 us; error 53 us; skew 500 ppm
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709034, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:34.769846 16999 init.cc:377] Logged in from keytab as oryx/127.9.236.1@KRBTEST.COM (short username oryx)
I20260929 19:10:34.770359 16999 webserver.cc:460] Webserver started at http://127.9.236.1:34031/ using document root <none> and password file <none>
I20260929 19:10:34.770582 16999 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:34.770648 16999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:34.772217 16999 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:34.772886 17102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:34.773080 16999 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:34.773198 16999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "56e2d743ca3e4d018b7e997ec34773c0"
format_stamp: "Formatted at 2026-09-29 19:10:33 on dist-test-slave-wftq"
server_key: "10bb7653e6ed3c2bc1e67e1b7a447c7b"
server_key_iv: "3869670918a7a76833a0e1e67a4832e9"
server_key_version: "encryptionkey@0"
I20260929 19:10:34.773578 16999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:34.786854 16999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:34.788949 16999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:34.789095 16999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:34.789394 16999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:34.789726 16999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:34.789759 16999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:34.789793 16999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:34.789813 16999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:34.797015 16999 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:46669
I20260929 19:10:34.797123 17215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:46669 every 8 connection(s)
I20260929 19:10:34.797466 16999 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709034, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.61@KRBTEST.COM
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709034, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709034, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.60@KRBTEST.COM
I20260929 19:10:34.807503 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 16999
I20260929 19:10:34.807663 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 16447
I20260929 19:10:34.809340 16900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.799161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:35357 (local address 127.9.236.61:43681)
0929 19:10:34.799203 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.799205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.799212 (+ 7us) server_negotiation.cc:407] Connection header received
0929 19:10:34.799453 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.799454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.799464 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.799493 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.800574 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.801395 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.802910 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.803191 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.803370 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.805982 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.805985 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.805985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.805995 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.807839 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.808308 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.808309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.808310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.808369 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.808627 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.808628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.808628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.808765 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.808807 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.809164 (+ 357us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.809252 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:10:34.809341 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.799037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:42041 (local address 127.9.236.62:38041)
0929 19:10:34.799126 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.799128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.799141 (+ 13us) server_negotiation.cc:407] Connection header received
0929 19:10:34.799450 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.799452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.799465 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.799517 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.801182 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.801862 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.802954 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.803154 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.803215 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.806430 (+ 3215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.806432 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.806432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.806440 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.808092 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.808436 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.808438 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.808438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.808496 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.808658 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.808658 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.808659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.808778 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.808833 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.809151 (+ 318us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.809236 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 19:10:34.809612 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.798347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.1:42041)
0929 19:10:34.798540 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.798594 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.799327 (+ 733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.799608 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.799613 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.800212 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:34.800467 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.800474 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.801958 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.801959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.802857 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.802858 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.803285 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.803286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.803359 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.804087 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:34.804090 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:34.806330 (+ 2240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:34.808183 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.808185 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.808185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.808372 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.808552 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.808553 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.808553 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.808612 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.808829 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:34.808838 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:34.809002 (+ 164us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.809129 (+ 127us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.809326 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":40,"thread_start_us":98,"threads_started":1}
I20260929 19:10:34.809705 17223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.798877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:43681 (local address 127.9.236.1:35357)
0929 19:10:34.799012 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.799023 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.799343 (+ 320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.799611 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.799615 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.800212 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:34.800466 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.800472 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.801515 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.801516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.802820 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.802823 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.803441 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.803442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.803518 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.804165 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:34.804171 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:34.805860 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:34.807924 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.807930 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.807933 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.808228 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.808438 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.808438 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.808439 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.808562 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.808849 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:34.808850 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:34.809005 (+ 155us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.809426 (+ 421us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.809544 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":50,"thread_start_us":88,"threads_started":1}
I20260929 19:10:34.810223 17217 heartbeater.cc:347] Connected to a master server at 127.9.236.62:38041
I20260929 19:10:34.810333 17217 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:34.810536 17217 heartbeater.cc:511] Master 127.9.236.62:38041 requested a full tablet report, sending...
I20260929 19:10:34.811115 16251 ts_manager.cc:194] Re-registered known tserver with Master: 56e2d743ca3e4d018b7e997ec34773c0 (127.9.236.1:46669)
I20260929 19:10:34.811559 16251 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:42041
I20260929 19:10:34.813812 17218 heartbeater.cc:347] Connected to a master server at 127.9.236.61:43681
I20260929 19:10:34.813879 17218 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:34.813952 17218 heartbeater.cc:511] Master 127.9.236.61:43681 requested a full tablet report, sending...
I20260929 19:10:34.814148 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:39003
--local_ip_for_outbound_sockets=127.9.236.2
--tserver_master_addrs=127.9.236.62:38041
--webserver_port=44295
--webserver_interface=127.9.236.2
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.2.keytab
--principal=oryx/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.236.62:38041,127.9.236.61:43681
--tserver_master_addrs=127.9.236.62:38041,127.9.236.61:43681,127.9.236.60:41353 with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:10:34.814370 16849 ts_manager.cc:194] Registered new tserver with Master: 56e2d743ca3e4d018b7e997ec34773c0 (127.9.236.1:46669)
I20260929 19:10:34.816239 17222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.798784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:53379 (local address 127.9.236.60:41353)
0929 19:10:34.798892 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.798895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.798908 (+ 13us) server_negotiation.cc:407] Connection header received
0929 19:10:34.799460 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.799462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.799479 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.799527 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.800608 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.801598 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.802784 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.803110 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.803187 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.806825 (+ 3638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.806826 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.806826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.806833 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.814867 (+ 8034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.815291 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.815292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.815293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.815367 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.815554 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.815555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.815555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.815714 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.815797 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.816025 (+ 228us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.816129 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":52,"threads_started":1}
I20260929 19:10:34.816601 17221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.798672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:41353 (local address 127.9.236.1:53379)
0929 19:10:34.798800 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.798811 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.799327 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.799618 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.799618 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.800217 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:34.800467 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.800474 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.801679 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.801680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.802703 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.802705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.803203 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.803204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.803359 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.804043 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:34.804056 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:34.806751 (+ 2695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:34.814986 (+ 8235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.814987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.814988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.815176 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.815436 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.815437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.815437 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.815489 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.815837 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:34.815839 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:34.815894 (+ 55us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.815988 (+ 94us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.816134 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":85,"thread_start_us":62,"threads_started":1}
I20260929 19:10:34.817090 17216 heartbeater.cc:347] Connected to a master server at 127.9.236.60:41353
I20260929 19:10:34.817175 17216 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:34.817272 17216 heartbeater.cc:511] Master 127.9.236.60:41353 requested a full tablet report, sending...
I20260929 19:10:34.817734 17037 ts_manager.cc:194] Registered new tserver with Master: 56e2d743ca3e4d018b7e997ec34773c0 (127.9.236.1:46669)
W20260929 19:10:34.897882 17224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:34.898084 17224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:34.898105 17224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:34.899247 17224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:34.899294 17224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:34.899329 17224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:10:34.900767 17224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.2.keytab
--principal=oryx/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:39003
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=44295
--tserver_master_addrs=127.9.236.62:38041,127.9.236.61:43681,127.9.236.60:41353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:34.900951 17224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:34.901146 17224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:34.905838 17229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:34.905843 17232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:34.905838 17230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:34.906136 17224 server_base.cc:1034] running on GCE node
I20260929 19:10:34.906364 17224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:34.906584 17224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:34.907742 17224 hybrid_clock.cc:648] HybridClock initialized: now 1790709034907725 us; error 37 us; skew 500 ppm
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709034, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:34.910106 17224 init.cc:377] Logged in from keytab as oryx/127.9.236.2@KRBTEST.COM (short username oryx)
I20260929 19:10:34.910526 17224 webserver.cc:460] Webserver started at http://127.9.236.2:44295/ using document root <none> and password file <none>
I20260929 19:10:34.910729 17224 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:34.910789 17224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:34.912237 17224 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:34.912889 17239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:34.913193 17224 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:34.913300 17224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "85ed2357da7247fc89f778b65e83aa92"
format_stamp: "Formatted at 2026-09-29 19:10:33 on dist-test-slave-wftq"
server_key: "6c1e5f755cf41e8a098b39ae2971af6b"
server_key_iv: "9fcade02521f44ccaf658ad5e63aa65d"
server_key_version: "encryptionkey@0"
I20260929 19:10:34.913659 17224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:34.934311 17224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:34.936322 17224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:34.936450 17224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:34.936658 17224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:34.936950 17224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:34.936983 17224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:34.937012 17224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:34.937028 17224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:34.943290 17224 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:39003
I20260929 19:10:34.943375 17352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:39003 every 8 connection(s)
I20260929 19:10:34.943661 17224 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:10:34.950426 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 17224
I20260929 19:10:34.950618 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 16581
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709034, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.60@KRBTEST.COM
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709034, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM
Sep 29 19:10:34 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709034, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.61@KRBTEST.COM
I20260929 19:10:34.954952 17222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.944788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:33845 (local address 127.9.236.60:41353)
0929 19:10:34.944840 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.944849 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.944994 (+ 145us) server_negotiation.cc:407] Connection header received
0929 19:10:34.945566 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.945568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.945583 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.945636 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.946640 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.947185 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.948656 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.948982 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.949043 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.951699 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.951701 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.951702 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.951709 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.953294 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.953833 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.953834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.953835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.953896 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.954165 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.954166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.954166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.954323 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.954365 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.954745 (+ 380us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.954845 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 19:10:34.955590 17359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.944706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:41353 (local address 127.9.236.2:33845)
0929 19:10:34.944901 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.944911 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.945385 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.945686 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.945687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.946293 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:34.946516 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.946554 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.947282 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.947282 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.948554 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.948556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.949129 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.949130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.949248 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.949681 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:34.949691 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:34.951607 (+ 1916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:34.953406 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.953414 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.953417 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.953733 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.953959 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.953960 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.953960 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.954098 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.954410 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:34.954417 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:34.954596 (+ 179us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.955122 (+ 526us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.955351 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":26,"thread_start_us":89,"threads_started":1}
I20260929 19:10:34.956058 17353 heartbeater.cc:347] Connected to a master server at 127.9.236.60:41353
I20260929 19:10:34.956163 17353 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:34.956373 17353 heartbeater.cc:511] Master 127.9.236.60:41353 requested a full tablet report, sending...
I20260929 19:10:34.956838 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.944507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:34543 (local address 127.9.236.62:38041)
0929 19:10:34.944565 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.944567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.944784 (+ 217us) server_negotiation.cc:407] Connection header received
0929 19:10:34.945458 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.945459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.945473 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.945523 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.946608 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.947426 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.949837 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.950107 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.950183 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.953590 (+ 3407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.953592 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.953592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.953598 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.955143 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.955748 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.955749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.955750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.955806 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.956026 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.956027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.956027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.956389 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.956449 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.956653 (+ 204us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.956723 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 19:10:34.956838 17037 ts_manager.cc:194] Registered new tserver with Master: 85ed2357da7247fc89f778b65e83aa92 (127.9.236.2:39003)
I20260929 19:10:34.957180 17358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.944400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.2:34543)
0929 19:10:34.944649 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.944691 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.945352 (+ 661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.945658 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.945663 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.946302 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:34.946513 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.946519 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.948657 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.948658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.949316 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.949318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.950219 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.950220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.950297 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.951830 (+ 1533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:34.951835 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:34.953125 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:34.955215 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.955216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.955216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.955482 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.955882 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.955883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.955883 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.955943 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.956486 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:34.956487 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:34.956538 (+ 51us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.956927 (+ 389us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.957048 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":84,"threads_started":1}
I20260929 19:10:34.957540 17354 heartbeater.cc:347] Connected to a master server at 127.9.236.62:38041
I20260929 19:10:34.957594 17354 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:34.957646 17354 heartbeater.cc:511] Master 127.9.236.62:38041 requested a full tablet report, sending...
I20260929 19:10:34.957736 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:43875
--local_ip_for_outbound_sockets=127.9.236.3
--tserver_master_addrs=127.9.236.62:38041
--webserver_port=32825
--webserver_interface=127.9.236.3
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.3.keytab
--principal=oryx/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.236.62:38041,127.9.236.61:43681
--tserver_master_addrs=127.9.236.62:38041,127.9.236.61:43681,127.9.236.60:41353 with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:10:34.957885 16251 ts_manager.cc:194] Re-registered known tserver with Master: 85ed2357da7247fc89f778b65e83aa92 (127.9.236.2:39003)
I20260929 19:10:34.958436 16251 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:34543
I20260929 19:10:34.958611 16900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.944457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:44585 (local address 127.9.236.61:43681)
0929 19:10:34.944507 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:34.944509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:34.944795 (+ 286us) server_negotiation.cc:407] Connection header received
0929 19:10:34.945528 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:34.945529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:34.945542 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:34.945604 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:34.946645 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.947421 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.948642 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.948939 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.949016 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.955295 (+ 6279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:34.955296 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:34.955297 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:34.955304 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:34.956936 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.957776 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.957778 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.957778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.957858 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:34.958072 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:34.958073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:34.958074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:34.958207 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:34.958256 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:34.958424 (+ 168us) server_negotiation.cc:299] Negotiation successful
0929 19:10:34.958511 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 19:10:34.958922 17357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:34.944437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:43681 (local address 127.9.236.2:44585)
0929 19:10:34.944708 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:34.944720 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:34.945385 (+ 665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:34.945658 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:34.945665 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:34.946269 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:34.946515 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.946522 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.947507 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.947508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.948554 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:34.948556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:34.949042 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:34.949043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:34.949223 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:34.949851 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:34.949856 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:34.954938 (+ 5082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:34.957376 (+ 2438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.957377 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.957377 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.957554 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.957941 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:34.957942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:34.957942 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:34.958003 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:34.958284 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:34.958285 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:34.958329 (+ 44us) client_negotiation.cc:769] Sending connection context
0929 19:10:34.958699 (+ 370us) client_negotiation.cc:240] Negotiation successful
0929 19:10:34.958810 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":18,"thread_start_us":127,"threads_started":1}
I20260929 19:10:34.959331 17355 heartbeater.cc:347] Connected to a master server at 127.9.236.61:43681
I20260929 19:10:34.959381 17355 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:34.959547 17355 heartbeater.cc:511] Master 127.9.236.61:43681 requested a full tablet report, sending...
I20260929 19:10:34.959837 16849 ts_manager.cc:194] Registered new tserver with Master: 85ed2357da7247fc89f778b65e83aa92 (127.9.236.2:39003)
W20260929 19:10:35.037545 17360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:35.037760 17360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:35.037789 17360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:35.038920 17360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:35.038967 17360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:35.039003 17360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:10:35.040432 17360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.3.keytab
--principal=oryx/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:43875
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=32825
--tserver_master_addrs=127.9.236.62:38041,127.9.236.61:43681,127.9.236.60:41353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:35.040660 17360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:35.040886 17360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:35.046382 17368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:35.046398 17365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:35.046388 17366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:35.046528 17360 server_base.cc:1034] running on GCE node
I20260929 19:10:35.046940 17360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:35.047145 17360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:35.048365 17360 hybrid_clock.cc:648] HybridClock initialized: now 1790709035048349 us; error 35 us; skew 500 ppm
Sep 29 19:10:35 dist-test-slave-wftq krb5kdc[16206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709035, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:35.055899 17360 init.cc:377] Logged in from keytab as oryx/127.9.236.3@KRBTEST.COM (short username oryx)
I20260929 19:10:35.056380 17360 webserver.cc:460] Webserver started at http://127.9.236.3:32825/ using document root <none> and password file <none>
I20260929 19:10:35.056589 17360 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:35.056646 17360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:35.058022 17360 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:35.058719 17375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:35.058956 17360 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:35.059062 17360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "4bff726d322940009260aeb6fc83fa15"
format_stamp: "Formatted at 2026-09-29 19:10:34 on dist-test-slave-wftq"
server_key: "c9fa7bf04857b4ace769a42f16f5432b"
server_key_iv: "d06b20216872eaf8a6d2d9c9f7adecc6"
server_key_version: "encryptionkey@0"
I20260929 19:10:35.059423 17360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:35.083580 17360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:35.085419 17360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:35.085542 17360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:35.085788 17360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:35.086081 17360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:35.086153 17360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:35.086200 17360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:35.086230 17360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:35.092479 17360 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:43875
I20260929 19:10:35.092538 17488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:43875 every 8 connection(s)
I20260929 19:10:35.092859 17360 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:10:35.093052 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 17360
Sep 29 19:10:35 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709035, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM
I20260929 19:10:35.102392 16997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.094206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51686 (local address 127.9.236.61:43681)
0929 19:10:35.094244 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:35.094246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:35.094290 (+ 44us) server_negotiation.cc:407] Connection header received
0929 19:10:35.094411 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:35.094413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:35.094430 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:35.094474 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:35.095325 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.096216 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.097306 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.097609 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.097647 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.099246 (+ 1599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:35.099247 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:35.099248 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:35.099254 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:35.100935 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:35.101347 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:35.101349 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:35.101349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:35.101412 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:35.101747 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:35.101748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:35.101748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:35.101896 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:35.101977 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:35.102195 (+ 218us) server_negotiation.cc:299] Negotiation successful
0929 19:10:35.102277 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 29 19:10:35 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709035, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.61@KRBTEST.COM
Sep 29 19:10:35 dist-test-slave-wftq krb5kdc[16206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709035, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.60@KRBTEST.COM
I20260929 19:10:35.104349 17501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.094285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34406 (local address 127.9.236.62:38041)
0929 19:10:35.094911 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:35.094913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:35.094920 (+ 7us) server_negotiation.cc:407] Connection header received
0929 19:10:35.094944 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:35.094944 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:35.094956 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:35.095021 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:35.095680 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.096537 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.097505 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.097801 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.097856 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.099745 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:35.099746 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:35.099746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:35.099753 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:35.101967 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:35.102319 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:35.102320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:35.102320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:35.102376 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:35.103431 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:35.103432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:35.103432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:35.103577 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:35.103672 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:35.104109 (+ 437us) server_negotiation.cc:299] Negotiation successful
0929 19:10:35.104240 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":84,"threads_started":1}
I20260929 19:10:35.107735 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.093871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:49817 (local address 127.9.236.62:38041)
0929 19:10:35.093928 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:35.093931 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:35.095280 (+ 1349us) server_negotiation.cc:407] Connection header received
0929 19:10:35.095359 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:35.095360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:35.095370 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:35.095394 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:35.096272 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.097051 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.098526 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.098818 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.098853 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.102548 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:35.102550 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:35.102550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:35.102557 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:35.104349 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:35.105966 (+ 1617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:35.105967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:35.105967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:35.106026 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:35.106671 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:35.106672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:35.106672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:35.106793 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:35.106838 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:35.107340 (+ 502us) server_negotiation.cc:299] Negotiation successful
0929 19:10:35.107444 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 19:10:35.107826 17222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.094334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:56777 (local address 127.9.236.60:41353)
0929 19:10:35.094390 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:35.094392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:35.097554 (+ 3162us) server_negotiation.cc:407] Connection header received
0929 19:10:35.097650 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:35.097651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:35.097664 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:35.097700 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:35.098104 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.098799 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.099796 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.100066 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.100104 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.103591 (+ 3487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:35.103592 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:35.103593 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:35.103599 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:35.105237 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:35.106996 (+ 1759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:35.106997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:35.106998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:35.107056 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:35.107279 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:35.107279 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:35.107280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:35.107399 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:35.107440 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:35.107650 (+ 210us) server_negotiation.cc:299] Negotiation successful
0929 19:10:35.107731 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 19:10:35.108044 17497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.094156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:41353 (local address 127.9.236.3:56777)
0929 19:10:35.097487 (+ 3331us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:35.097498 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:35.097581 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:35.097773 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:35.097774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:35.097958 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:35.098021 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:35.098023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.098885 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.098886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:35.099649 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:35.099652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.100178 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.100179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:35.100236 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.100558 (+ 322us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:35.100563 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:35.103504 (+ 2941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:35.105342 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:35.105344 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:35.105344 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:35.106219 (+ 875us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:35.107127 (+ 908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:35.107128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:35.107128 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:35.107206 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:35.107490 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:35.107491 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:35.107541 (+ 50us) client_negotiation.cc:769] Sending connection context
0929 19:10:35.107622 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 19:10:35.107812 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3312,"thread_start_us":64,"threads_started":1}
I20260929 19:10:35.106030 16251 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34406:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:10:35.108654 16251 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:10:35.108995 17500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.093755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.3:49817)
0929 19:10:35.095222 (+ 1467us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:35.095231 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:35.095302 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:35.095440 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:35.095440 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:35.095861 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:35.096168 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:35.096174 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.097193 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.097194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:35.098436 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:35.098439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.098937 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.098938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:35.099068 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.100072 (+ 1004us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:35.100082 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:35.102174 (+ 2092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:35.104877 (+ 2703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:35.104889 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:35.104892 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:35.105886 (+ 994us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:35.106097 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:35.106098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:35.106098 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:35.106592 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:35.106867 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:35.106873 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:35.107184 (+ 311us) client_negotiation.cc:769] Sending connection context
0929 19:10:35.108756 (+ 1572us) client_negotiation.cc:240] Negotiation successful
0929 19:10:35.108889 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1437,"mutex_wait_us":227,"thread_start_us":65,"threads_started":1}
I20260929 19:10:35.109475 16900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.093774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:58481 (local address 127.9.236.61:43681)
0929 19:10:35.093824 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:35.093826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:35.094471 (+ 645us) server_negotiation.cc:407] Connection header received
0929 19:10:35.095163 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:35.095164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:35.095175 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:35.095216 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:35.096272 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.097039 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.100697 (+ 3658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.100970 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.101011 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.105455 (+ 4444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:35.105456 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:35.105456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:35.105463 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:35.106899 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:35.108663 (+ 1764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:35.108665 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:35.108665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:35.108719 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:35.108905 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:35.108906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:35.108907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:35.109033 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:35.109071 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:35.109314 (+ 243us) server_negotiation.cc:299] Negotiation successful
0929 19:10:35.109385 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:10:35.109862 17489 heartbeater.cc:347] Connected to a master server at 127.9.236.60:41353
I20260929 19:10:35.109973 17489 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:35.110198 17489 heartbeater.cc:511] Master 127.9.236.60:41353 requested a full tablet report, sending...
I20260929 19:10:35.110399 17504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.094655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33456 (local address 127.9.236.60:41353)
0929 19:10:35.100405 (+ 5750us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:35.100407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:35.100414 (+ 7us) server_negotiation.cc:407] Connection header received
0929 19:10:35.100432 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:35.100433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:35.100444 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:35.103929 (+ 3485us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:35.103962 (+ 33us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.104628 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.105712 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.105987 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.106297 (+ 310us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.107294 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:35.107295 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:35.107295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:35.107301 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:35.109045 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:35.109446 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:35.109446 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:35.109447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:35.109501 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:35.109767 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:35.109768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:35.109768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:35.109931 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:35.110081 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:35.110225 (+ 144us) server_negotiation.cc:299] Negotiation successful
0929 19:10:35.110314 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5730,"thread_start_us":77,"threads_started":1}
I20260929 19:10:35.110854 17498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.093755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:43681 (local address 127.9.236.3:58481)
0929 19:10:35.094331 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:35.094387 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:35.095053 (+ 666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:35.095394 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:35.095398 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:35.095951 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:35.096168 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:35.096174 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.097197 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.097197 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:35.099945 (+ 2748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:35.099947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.101090 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.101091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:35.101193 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.101543 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:35.101548 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:35.104538 (+ 2990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:35.108380 (+ 3842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:35.108381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:35.108381 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:35.108576 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:35.108787 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:35.108787 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:35.108788 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:35.108841 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:35.109139 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:35.109140 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:35.109198 (+ 58us) client_negotiation.cc:769] Sending connection context
0929 19:10:35.110538 (+ 1340us) client_negotiation.cc:240] Negotiation successful
0929 19:10:35.110633 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":45,"thread_start_us":114,"threads_started":1}
I20260929 19:10:35.111413 17037 ts_manager.cc:194] Registered new tserver with Master: 4bff726d322940009260aeb6fc83fa15 (127.9.236.3:43875)
I20260929 19:10:35.111871 17491 heartbeater.cc:347] Connected to a master server at 127.9.236.61:43681
I20260929 19:10:35.111932 17491 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:35.112017 17491 heartbeater.cc:511] Master 127.9.236.61:43681 requested a full tablet report, sending...
I20260929 19:10:35.112368 17490 heartbeater.cc:347] Connected to a master server at 127.9.236.62:38041
I20260929 19:10:35.112413 17490 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:35.112468 17490 heartbeater.cc:511] Master 127.9.236.62:38041 requested a full tablet report, sending...
I20260929 19:10:35.112474 16849 ts_manager.cc:194] Registered new tserver with Master: 4bff726d322940009260aeb6fc83fa15 (127.9.236.3:43875)
I20260929 19:10:35.112707 16251 ts_manager.cc:194] Re-registered known tserver with Master: 4bff726d322940009260aeb6fc83fa15 (127.9.236.3:43875)
I20260929 19:10:35.113281 16251 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:49817
I20260929 19:10:35.118264 17510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.114113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51602 (local address 127.9.236.1:46669)
0929 19:10:35.114562 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:35.114566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:35.114574 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:10:35.114609 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:35.114611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:35.114628 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:35.114849 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:35.114894 (+ 45us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.115988 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.117199 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.117896 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.117977 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.118053 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:35.118137 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 19:10:35.118175 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":99,"threads_started":1}
I20260929 19:10:35.118278 16809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.114005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:46669 (local address 127.0.0.1:51602)
0929 19:10:35.114293 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:35.114303 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:35.114347 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:35.114736 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:35.114737 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:35.114739 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:35.114786 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:35.114787 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.116088 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.116089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:35.117065 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:35.117068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.117986 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.117987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:35.118052 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.118057 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:10:35.118123 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 19:10:35.118187 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270}
I20260929 19:10:35.118808 17509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.114101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:43875 (local address 127.0.0.1:44808)
0929 19:10:35.114429 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:35.114436 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:35.114469 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:35.114871 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:35.114872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:35.114874 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:35.114908 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:35.114909 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.116154 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.116155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:35.117328 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:35.117329 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.118197 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.118198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:35.118254 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.118258 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:10:35.118299 (+ 41us) client_negotiation.cc:240] Negotiation successful
0929 19:10:35.118748 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":18,"thread_start_us":149,"threads_started":1}
I20260929 19:10:35.118983 17512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.114326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46566 (local address 127.9.236.2:39003)
0929 19:10:35.114689 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:35.114695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:35.114707 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:10:35.114754 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:35.114757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:35.114776 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:35.114870 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:35.115081 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.115793 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.117414 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.118178 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.118256 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.118576 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:35.118861 (+ 285us) server_negotiation.cc:299] Negotiation successful
0929 19:10:35.118905 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":122,"threads_started":1}
I20260929 19:10:35.119092 17513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.114495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44808 (local address 127.9.236.3:43875)
0929 19:10:35.114680 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:35.114683 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:35.114692 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:35.114723 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:35.114725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:35.114738 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:35.114805 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:35.114976 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.116051 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.117416 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.118008 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.118077 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.118164 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:35.118921 (+ 757us) server_negotiation.cc:299] Negotiation successful
0929 19:10:35.118974 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":92,"threads_started":1}
I20260929 19:10:35.119167 17511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.114069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:39003 (local address 127.0.0.1:46566)
0929 19:10:35.114297 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:35.114307 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:35.114347 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:35.114941 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:35.114943 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:35.114945 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:35.115007 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:35.115009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.115876 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.115877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:35.117306 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:35.117309 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.118613 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.118614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:35.118677 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.118682 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:10:35.118734 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 19:10:35.118795 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":32,"thread_start_us":112,"threads_started":1}
I20260929 19:10:35.119158 17150 tablet_service.cc:1467] Processing CreateTablet for tablet 99de3c00fd8141b4b2550f5c0f7a326a (DEFAULT_TABLE table=test-table [id=7248c95dd8a040f5b555f207e6ecf451]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:35.119580 17150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99de3c00fd8141b4b2550f5c0f7a326a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:35.119835 17423 tablet_service.cc:1467] Processing CreateTablet for tablet 99de3c00fd8141b4b2550f5c0f7a326a (DEFAULT_TABLE table=test-table [id=7248c95dd8a040f5b555f207e6ecf451]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:35.120131 17423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99de3c00fd8141b4b2550f5c0f7a326a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:35.119997 17287 tablet_service.cc:1467] Processing CreateTablet for tablet 99de3c00fd8141b4b2550f5c0f7a326a (DEFAULT_TABLE table=test-table [id=7248c95dd8a040f5b555f207e6ecf451]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:35.120380 17287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99de3c00fd8141b4b2550f5c0f7a326a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:35.123356 17514 tablet_bootstrap.cc:492] T 99de3c00fd8141b4b2550f5c0f7a326a P 56e2d743ca3e4d018b7e997ec34773c0: Bootstrap starting.
I20260929 19:10:35.123988 17514 tablet_bootstrap.cc:654] T 99de3c00fd8141b4b2550f5c0f7a326a P 56e2d743ca3e4d018b7e997ec34773c0: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:35.124245 17514 log.cc:824] T 99de3c00fd8141b4b2550f5c0f7a326a P 56e2d743ca3e4d018b7e997ec34773c0: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:35.124537 17515 tablet_bootstrap.cc:492] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15: Bootstrap starting.
I20260929 19:10:35.125077 17515 tablet_bootstrap.cc:654] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:35.125128 17514 tablet_bootstrap.cc:492] T 99de3c00fd8141b4b2550f5c0f7a326a P 56e2d743ca3e4d018b7e997ec34773c0: No bootstrap required, opened a new log
I20260929 19:10:35.125196 17514 ts_tablet_manager.cc:1397] T 99de3c00fd8141b4b2550f5c0f7a326a P 56e2d743ca3e4d018b7e997ec34773c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:10:35.125361 17516 tablet_bootstrap.cc:492] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92: Bootstrap starting.
I20260929 19:10:35.125392 17515 log.cc:824] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:35.126071 17516 tablet_bootstrap.cc:654] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:35.126291 17515 tablet_bootstrap.cc:492] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15: No bootstrap required, opened a new log
I20260929 19:10:35.126358 17515 ts_tablet_manager.cc:1397] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:10:35.126410 17516 log.cc:824] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:35.127310 17516 tablet_bootstrap.cc:492] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92: No bootstrap required, opened a new log
I20260929 19:10:35.127360 17516 ts_tablet_manager.cc:1397] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:10:35.127455 17514 raft_consensus.cc:348] T 99de3c00fd8141b4b2550f5c0f7a326a P 56e2d743ca3e4d018b7e997ec34773c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 } } peers { permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 } } peers { permanent_uuid: "4bff726d322940009260aeb6fc83fa15" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43875 } }
I20260929 19:10:35.127599 17514 raft_consensus.cc:374] T 99de3c00fd8141b4b2550f5c0f7a326a P 56e2d743ca3e4d018b7e997ec34773c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:35.127619 17514 raft_consensus.cc:729] T 99de3c00fd8141b4b2550f5c0f7a326a P 56e2d743ca3e4d018b7e997ec34773c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e2d743ca3e4d018b7e997ec34773c0, State: Initialized, Role: FOLLOWER
I20260929 19:10:35.127710 17514 consensus_queue.cc:260] T 99de3c00fd8141b4b2550f5c0f7a326a P 56e2d743ca3e4d018b7e997ec34773c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 } } peers { permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 } } peers { permanent_uuid: "4bff726d322940009260aeb6fc83fa15" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43875 } }
I20260929 19:10:35.127931 17514 ts_tablet_manager.cc:1428] T 99de3c00fd8141b4b2550f5c0f7a326a P 56e2d743ca3e4d018b7e997ec34773c0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:10:35.128134 17217 heartbeater.cc:503] Master 127.9.236.62:38041 was elected leader, sending a full tablet report...
I20260929 19:10:35.128455 17515 raft_consensus.cc:348] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 } } peers { permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 } } peers { permanent_uuid: "4bff726d322940009260aeb6fc83fa15" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43875 } }
I20260929 19:10:35.128626 17515 raft_consensus.cc:374] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:35.128665 17515 raft_consensus.cc:729] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bff726d322940009260aeb6fc83fa15, State: Initialized, Role: FOLLOWER
I20260929 19:10:35.128794 17515 consensus_queue.cc:260] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 } } peers { permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 } } peers { permanent_uuid: "4bff726d322940009260aeb6fc83fa15" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43875 } }
I20260929 19:10:35.129071 17515 ts_tablet_manager.cc:1428] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:10:35.129386 17516 raft_consensus.cc:348] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 } } peers { permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 } } peers { permanent_uuid: "4bff726d322940009260aeb6fc83fa15" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43875 } }
I20260929 19:10:35.129520 17516 raft_consensus.cc:374] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:35.129549 17516 raft_consensus.cc:729] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85ed2357da7247fc89f778b65e83aa92, State: Initialized, Role: FOLLOWER
I20260929 19:10:35.129654 17516 consensus_queue.cc:260] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 } } peers { permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 } } peers { permanent_uuid: "4bff726d322940009260aeb6fc83fa15" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43875 } }
I20260929 19:10:35.129884 17516 ts_tablet_manager.cc:1428] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260929 19:10:35.129971 17354 heartbeater.cc:503] Master 127.9.236.62:38041 was elected leader, sending a full tablet report...
I20260929 19:10:35.131110 17490 heartbeater.cc:503] Master 127.9.236.62:38041 was elected leader, sending a full tablet report...
W20260929 19:10:35.194860 17356 tablet.cc:2369] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:10:35.209232 17504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.206504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33462 (local address 127.9.236.60:41353)
0929 19:10:35.206557 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:35.206559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:35.206572 (+ 13us) server_negotiation.cc:407] Connection header received
0929 19:10:35.206670 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:35.206670 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:35.206697 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:35.206776 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:35.206937 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.207447 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.208383 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.208795 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.208870 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.208924 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:35.209088 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 19:10:35.209148 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 19:10:35.209256 17511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.206416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:41353 (local address 127.0.0.1:33462)
0929 19:10:35.206513 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:35.206527 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:35.206586 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:35.206809 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:35.206810 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:35.206812 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:35.206878 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:35.206881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.207518 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.207519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:35.208292 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:35.208293 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.208907 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.208909 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:35.209006 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.209011 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:10:35.209072 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 19:10:35.209154 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260929 19:10:35.209722 17047 raft_consensus.cc:1203] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260929 19:10:35.210562 17076 sys_catalog.cc:455] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64eaa0faf742494680dadbf96bb413f0. Latest consensus state: current_term: 1 leader_uuid: "64eaa0faf742494680dadbf96bb413f0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: NON_VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: true } } }
I20260929 19:10:35.210704 17076 sys_catalog.cc:458] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [sys.catalog]: This master's current role is: LEARNER
I20260929 19:10:35.230294 17521 raft_consensus.cc:482] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:10:35.230428 17521 raft_consensus.cc:504] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 } } peers { permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 } } peers { permanent_uuid: "4bff726d322940009260aeb6fc83fa15" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43875 } }
I20260929 19:10:35.230917 17521 leader_election.cc:288] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56e2d743ca3e4d018b7e997ec34773c0 (127.9.236.1:46669), 85ed2357da7247fc89f778b65e83aa92 (127.9.236.2:39003)
I20260929 19:10:35.234977 17512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.231205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:37935 (local address 127.9.236.2:39003)
0929 19:10:35.231260 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:35.231262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:35.231276 (+ 14us) server_negotiation.cc:407] Connection header received
0929 19:10:35.231362 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:35.231363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:35.231376 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:35.231453 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:35.231672 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.232448 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.234017 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.234599 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.234676 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.234686 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:35.234867 (+ 181us) server_negotiation.cc:299] Negotiation successful
0929 19:10:35.234904 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 19:10:35.234977 17510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.231212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:43131 (local address 127.9.236.1:46669)
0929 19:10:35.231271 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:35.231273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:35.231346 (+ 73us) server_negotiation.cc:407] Connection header received
0929 19:10:35.231383 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:35.231384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:35.231395 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:35.231456 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:35.231672 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.232410 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.234017 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.234598 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.234676 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.234686 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:35.234867 (+ 181us) server_negotiation.cc:299] Negotiation successful
0929 19:10:35.234904 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 19:10:35.235056 17498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.231121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:39003 (local address 127.9.236.3:37935)
0929 19:10:35.231234 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:35.231248 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:35.231294 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:35.231467 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:35.231468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:35.231470 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:35.231599 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:35.231601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.232525 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.232526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:35.233912 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:35.233915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.234690 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.234691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:35.234785 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.234792 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 19:10:35.234852 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 19:10:35.234912 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":97}
I20260929 19:10:35.235056 17500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.231121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:46669 (local address 127.9.236.3:43131)
0929 19:10:35.231286 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:35.231295 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:35.231337 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:35.231504 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:35.231505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:35.231506 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:35.231599 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:35.231601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.232525 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.232526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:35.233912 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:35.233915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.234695 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.234695 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:35.234785 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.234792 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 19:10:35.234852 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 19:10:35.234912 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":190}
I20260929 19:10:35.235495 17170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99de3c00fd8141b4b2550f5c0f7a326a" candidate_uuid: "4bff726d322940009260aeb6fc83fa15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e2d743ca3e4d018b7e997ec34773c0" is_pre_election: true
I20260929 19:10:35.235579 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99de3c00fd8141b4b2550f5c0f7a326a" candidate_uuid: "4bff726d322940009260aeb6fc83fa15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85ed2357da7247fc89f778b65e83aa92" is_pre_election: true
I20260929 19:10:35.235674 17170 raft_consensus.cc:2454] T 99de3c00fd8141b4b2550f5c0f7a326a P 56e2d743ca3e4d018b7e997ec34773c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bff726d322940009260aeb6fc83fa15 in term 0.
I20260929 19:10:35.235751 17307 raft_consensus.cc:2454] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bff726d322940009260aeb6fc83fa15 in term 0.
I20260929 19:10:35.235934 17378 leader_election.cc:302] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bff726d322940009260aeb6fc83fa15, 56e2d743ca3e4d018b7e997ec34773c0; no voters:
I20260929 19:10:35.236130 17521 raft_consensus.cc:2781] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:10:35.236177 17521 raft_consensus.cc:482] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:10:35.236199 17521 raft_consensus.cc:3037] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:35.237042 17521 raft_consensus.cc:504] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 } } peers { permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 } } peers { permanent_uuid: "4bff726d322940009260aeb6fc83fa15" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43875 } }
I20260929 19:10:35.237243 17521 leader_election.cc:288] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 [CANDIDATE]: Term 1 election: Requested vote from peers 56e2d743ca3e4d018b7e997ec34773c0 (127.9.236.1:46669), 85ed2357da7247fc89f778b65e83aa92 (127.9.236.2:39003)
I20260929 19:10:35.237406 17170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99de3c00fd8141b4b2550f5c0f7a326a" candidate_uuid: "4bff726d322940009260aeb6fc83fa15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e2d743ca3e4d018b7e997ec34773c0"
I20260929 19:10:35.237428 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99de3c00fd8141b4b2550f5c0f7a326a" candidate_uuid: "4bff726d322940009260aeb6fc83fa15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85ed2357da7247fc89f778b65e83aa92"
I20260929 19:10:35.237495 17170 raft_consensus.cc:3037] T 99de3c00fd8141b4b2550f5c0f7a326a P 56e2d743ca3e4d018b7e997ec34773c0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:35.237501 17307 raft_consensus.cc:3037] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:35.238242 17170 raft_consensus.cc:2454] T 99de3c00fd8141b4b2550f5c0f7a326a P 56e2d743ca3e4d018b7e997ec34773c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bff726d322940009260aeb6fc83fa15 in term 1.
I20260929 19:10:35.238268 17307 raft_consensus.cc:2454] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bff726d322940009260aeb6fc83fa15 in term 1.
I20260929 19:10:35.238394 17378 leader_election.cc:302] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bff726d322940009260aeb6fc83fa15, 56e2d743ca3e4d018b7e997ec34773c0; no voters:
I20260929 19:10:35.238500 17521 raft_consensus.cc:2781] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:35.238693 17521 raft_consensus.cc:686] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 [term 1 LEADER]: Becoming Leader. State: Replica: 4bff726d322940009260aeb6fc83fa15, State: Running, Role: LEADER
I20260929 19:10:35.238798 17521 consensus_queue.cc:237] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 } } peers { permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 } } peers { permanent_uuid: "4bff726d322940009260aeb6fc83fa15" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43875 } }
I20260929 19:10:35.240017 16251 catalog_manager.cc:5495] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bff726d322940009260aeb6fc83fa15 (127.9.236.3). New cstate: current_term: 1 leader_uuid: "4bff726d322940009260aeb6fc83fa15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bff726d322940009260aeb6fc83fa15" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43875 } health_report { overall_health: HEALTHY } } }
I20260929 19:10:35.240581 16811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "64eaa0faf742494680dadbf96bb413f0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: NON_VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: true } } }
I20260929 19:10:35.240696 16811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:35.241365 17001 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0: attempting to promote NON_VOTER fde63ef43d8a440e96811f6d118232f2 to VOTER
I20260929 19:10:35.241575 17001 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false } }
I20260929 19:10:35.242122 16858 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 1 FOLLOWER]: Refusing update from remote peer 64eaa0faf742494680dadbf96bb413f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260929 19:10:35.242194 17047 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 1 LEARNER]: Refusing update from remote peer 64eaa0faf742494680dadbf96bb413f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260929 19:10:35.242389 17529 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260929 19:10:35.242517 17529 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260929 19:10:35.243165 16811 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, fde63ef43d8a440e96811f6d118232f2 (127.9.236.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false } } }
I20260929 19:10:35.243311 16858 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, fde63ef43d8a440e96811f6d118232f2 (127.9.236.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false } } }
I20260929 19:10:35.244200 16811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "64eaa0faf742494680dadbf96bb413f0" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false } } }
I20260929 19:10:35.244274 16811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:35.244287 17529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "64eaa0faf742494680dadbf96bb413f0" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false } } }
I20260929 19:10:35.244405 17529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:35.244292 17001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "64eaa0faf742494680dadbf96bb413f0" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false } } }
I20260929 19:10:35.244545 17001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:35.244693 17047 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, fde63ef43d8a440e96811f6d118232f2 (127.9.236.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false } } }
I20260929 19:10:35.245196 17530 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "64eaa0faf742494680dadbf96bb413f0" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false } } }
I20260929 19:10:35.245289 17530 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:10:35.245709 17076 sys_catalog.cc:455] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "64eaa0faf742494680dadbf96bb413f0" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false } } }
I20260929 19:10:35.245824 17076 sys_catalog.cc:458] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:10:35.264098 17513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.261539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44820 (local address 127.9.236.3:43875)
0929 19:10:35.261614 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:35.261615 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:35.261625 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:10:35.261660 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:35.261661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:35.261682 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:35.261752 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:35.261912 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.262457 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.263377 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.263565 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.263613 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.263760 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:35.263866 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:35.263927 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:35.264021 (+ 94us) server_negotiation.cc:299] Negotiation successful
0929 19:10:35.264038 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 19:10:35.266086 17307 raft_consensus.cc:1261] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92 [term 1 FOLLOWER]: Refusing update from remote peer 4bff726d322940009260aeb6fc83fa15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:35.266125 17170 raft_consensus.cc:1261] T 99de3c00fd8141b4b2550f5c0f7a326a P 56e2d743ca3e4d018b7e997ec34773c0 [term 1 FOLLOWER]: Refusing update from remote peer 4bff726d322940009260aeb6fc83fa15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:35.266469 17526 consensus_queue.cc:1035] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:35.266620 17521 consensus_queue.cc:1035] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:35.269236 17532 mvcc.cc:205] Tried to move back new op lower bound from 7334744208447135744 to 7334744208338939904. Current Snapshot: MvccSnapshot[applied={T|T < 7334744208447135744}]
I20260929 19:10:35.269580 17533 mvcc.cc:205] Tried to move back new op lower bound from 7334744208447135744 to 7334744208338939904. Current Snapshot: MvccSnapshot[applied={T|T < 7334744208447135744}]
I20260929 19:10:35.270521 17534 mvcc.cc:205] Tried to move back new op lower bound from 7334744208447135744 to 7334744208338939904. Current Snapshot: MvccSnapshot[applied={T|T < 7334744208447135744}]
I20260929 19:10:35.273659 16249 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:35.273756 16249 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:35.275420 16249 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64eaa0faf742494680dadbf96bb413f0: Sending DeleteTablet for 3 replicas of tablet 99de3c00fd8141b4b2550f5c0f7a326a
I20260929 19:10:35.275791 17150 tablet_service.cc:1514] Processing DeleteTablet for tablet 99de3c00fd8141b4b2550f5c0f7a326a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:35 UTC) from {username='oryx', principal='oryx/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:51602
I20260929 19:10:35.275882 17423 tablet_service.cc:1514] Processing DeleteTablet for tablet 99de3c00fd8141b4b2550f5c0f7a326a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:35 UTC) from {username='oryx', principal='oryx/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:44808
I20260929 19:10:35.276161 17287 tablet_service.cc:1514] Processing DeleteTablet for tablet 99de3c00fd8141b4b2550f5c0f7a326a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:35 UTC) from {username='oryx', principal='oryx/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:46566
I20260929 19:10:35.276782 17543 tablet_replica.cc:330] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15: stopping tablet replica
I20260929 19:10:35.276921 17543 raft_consensus.cc:2229] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:10:35.276961 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 16219
I20260929 19:10:35.277058 17543 raft_consensus.cc:2258] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:35.277323 17545 tablet_replica.cc:330] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92: stopping tablet replica
I20260929 19:10:35.277406 17545 raft_consensus.cc:2229] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:35.277510 17545 raft_consensus.cc:2258] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:35.277653 17542 tablet_replica.cc:330] T 99de3c00fd8141b4b2550f5c0f7a326a P 56e2d743ca3e4d018b7e997ec34773c0: stopping tablet replica
I20260929 19:10:35.277571 17543 ts_tablet_manager.cc:1905] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:35.277757 17542 raft_consensus.cc:2229] T 99de3c00fd8141b4b2550f5c0f7a326a P 56e2d743ca3e4d018b7e997ec34773c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:35.277868 17542 raft_consensus.cc:2258] T 99de3c00fd8141b4b2550f5c0f7a326a P 56e2d743ca3e4d018b7e997ec34773c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:35.278120 17545 ts_tablet_manager.cc:1905] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:35.278260 17542 ts_tablet_manager.cc:1905] T 99de3c00fd8141b4b2550f5c0f7a326a P 56e2d743ca3e4d018b7e997ec34773c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:35.279856 17542 ts_tablet_manager.cc:1918] T 99de3c00fd8141b4b2550f5c0f7a326a P 56e2d743ca3e4d018b7e997ec34773c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:10:35.279937 17542 log.cc:1196] T 99de3c00fd8141b4b2550f5c0f7a326a P 56e2d743ca3e4d018b7e997ec34773c0: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-0/wal/wals/99de3c00fd8141b4b2550f5c0f7a326a
I20260929 19:10:35.280122 17545 ts_tablet_manager.cc:1918] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:10:35.280210 17542 ts_tablet_manager.cc:1939] T 99de3c00fd8141b4b2550f5c0f7a326a P 56e2d743ca3e4d018b7e997ec34773c0: Deleting consensus metadata
I20260929 19:10:35.280297 17545 log.cc:1196] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/99de3c00fd8141b4b2550f5c0f7a326a
I20260929 19:10:35.280616 17545 ts_tablet_manager.cc:1939] T 99de3c00fd8141b4b2550f5c0f7a326a P 85ed2357da7247fc89f778b65e83aa92: Deleting consensus metadata
I20260929 19:10:35.282013 17543 ts_tablet_manager.cc:1918] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:10:35.282083 17543 log.cc:1196] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/99de3c00fd8141b4b2550f5c0f7a326a
I20260929 19:10:35.282315 17543 ts_tablet_manager.cc:1939] T 99de3c00fd8141b4b2550f5c0f7a326a P 4bff726d322940009260aeb6fc83fa15: Deleting consensus metadata
W20260929 19:10:35.284006 17376 connection.cc:535] server connection from 127.0.0.1:44808 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 19:10:35.284338 17241 connection.cc:535] server connection from 127.0.0.1:46566 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 19:10:35.284507 17105 connection.cc:535] server connection from 127.0.0.1:51602 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 19:10:35.290043 17504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.276577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33472 (local address 127.9.236.60:41353)
0929 19:10:35.276615 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:35.276617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:35.276626 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:35.279250 (+ 2624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:35.279252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:35.279266 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:35.279306 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:35.281934 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.282745 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.283845 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.284125 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.284193 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.286980 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:35.286982 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:35.286982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:35.286989 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:35.288529 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:35.288891 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:35.288893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:35.288893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:35.288945 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:35.289160 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:35.289161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:35.289161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:35.289288 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:35.289379 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:35.289848 (+ 469us) server_negotiation.cc:299] Negotiation successful
0929 19:10:35.289933 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:10:35.293728 16900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.285819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51688 (local address 127.9.236.61:43681)
0929 19:10:35.285879 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:35.285881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:35.286180 (+ 299us) server_negotiation.cc:407] Connection header received
0929 19:10:35.286341 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:35.286342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:35.286355 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:35.286384 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:35.287061 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.287850 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.288889 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.289214 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.289281 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.290928 (+ 1647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:35.290929 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:35.290930 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:35.290936 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:35.292478 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:35.292885 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:35.292886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:35.292886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:35.292936 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:35.293153 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:35.293154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:35.293154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:35.293279 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:35.293349 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:35.293547 (+ 198us) server_negotiation.cc:299] Negotiation successful
0929 19:10:35.293619 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 19:10:35.294685 17222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:35.286207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33484 (local address 127.9.236.60:41353)
0929 19:10:35.286261 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:35.286263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:35.286469 (+ 206us) server_negotiation.cc:407] Connection header received
0929 19:10:35.286604 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:35.286605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:35.286633 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:35.286662 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:35.289210 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.289877 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.290935 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:35.291137 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:35.291185 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:35.292268 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:35.292270 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:35.292270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:35.292276 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:35.293600 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:35.293936 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:35.293937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:35.293938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:35.293997 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:35.294175 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:35.294175 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:35.294176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:35.294278 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:35.294362 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:35.294529 (+ 167us) server_negotiation.cc:299] Negotiation successful
0929 19:10:35.294594 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 19:10:36.271739 17553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:36.271393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.1:54075)
0929 19:10:36.271589 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:36.271679 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":65,"threads_started":1}
I20260929 19:10:36.271745 17554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:36.271431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.3:49457)
0929 19:10:36.271552 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:36.271673 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":46,"threads_started":1}
W20260929 19:10:36.271973 17490 heartbeater.cc:644] Failed to heartbeat to 127.9.236.62:38041 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
W20260929 19:10:36.271996 17217 heartbeater.cc:644] Failed to heartbeat to 127.9.236.62:38041 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
I20260929 19:10:36.272084 17555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:36.271768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.2:35513)
0929 19:10:36.271969 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:36.272039 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":73,"threads_started":1}
W20260929 19:10:36.272246 17354 heartbeater.cc:644] Failed to heartbeat to 127.9.236.62:38041 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
I20260929 19:10:36.776173 17556 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64eaa0faf742494680dadbf96bb413f0)
I20260929 19:10:36.776353 17556 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false } }
I20260929 19:10:36.776641 17556 leader_election.cc:288] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64eaa0faf742494680dadbf96bb413f0 (127.9.236.62:38041), fde63ef43d8a440e96811f6d118232f2 (127.9.236.60:41353)
I20260929 19:10:36.776978 17558 raft_consensus.cc:482] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 64eaa0faf742494680dadbf96bb413f0)
I20260929 19:10:36.777125 17557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:36.776840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.0.0.1:34894)
0929 19:10:36.776971 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:36.777033 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":59,"threads_started":1}
W20260929 19:10:36.777312 16836 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
I20260929 19:10:36.777338 17558 raft_consensus.cc:504] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false } }
I20260929 19:10:36.777737 17557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:36.777643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.0.0.1:34900)
0929 19:10:36.777687 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:36.777714 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260929 19:10:36.777855 16836 leader_election.cc:334] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 64eaa0faf742494680dadbf96bb413f0 (127.9.236.62:38041): Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
I20260929 19:10:36.777932 17558 leader_election.cc:288] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64eaa0faf742494680dadbf96bb413f0 (127.9.236.62:38041), f0c0560533814a7cbebc5c5940f38a3d (127.9.236.61:43681)
I20260929 19:10:36.777987 17562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:36.777658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.0.0.1:34908)
0929 19:10:36.777790 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:36.777836 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":58,"threads_started":1}
W20260929 19:10:36.778326 17024 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
I20260929 19:10:36.778851 17562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:36.778734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.0.0.1:34924)
0929 19:10:36.778789 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:36.778824 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260929 19:10:36.779415 17024 leader_election.cc:334] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 64eaa0faf742494680dadbf96bb413f0 (127.9.236.62:38041): Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
I20260929 19:10:36.780776 17560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:36.777285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33490 (local address 127.9.236.60:41353)
0929 19:10:36.777377 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:36.777379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:36.777391 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:10:36.777418 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:36.777419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:36.777431 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:36.777527 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:36.777787 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:36.778834 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:36.780062 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:36.780467 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:36.780526 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:36.780539 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:36.780665 (+ 126us) server_negotiation.cc:299] Negotiation successful
0929 19:10:36.780716 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71,"thread_start_us":47,"threads_started":1}
I20260929 19:10:36.780783 17559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:36.777026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:41353 (local address 127.0.0.1:33490)
0929 19:10:36.777152 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:36.777162 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:36.777229 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:36.777594 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:36.777595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:36.777599 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:36.777659 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:36.777661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:36.778962 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:36.778963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:36.779979 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:36.779982 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:36.780538 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:36.780539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:36.780605 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:36.780610 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:10:36.780656 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 19:10:36.780712 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":60,"threads_started":1}
I20260929 19:10:36.780903 17564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:36.777979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51696 (local address 127.9.236.61:43681)
0929 19:10:36.778147 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:36.778149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:36.778159 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:10:36.778181 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:36.778182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:36.778193 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:36.778270 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:36.778449 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:36.779162 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:36.780205 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:36.780601 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:36.780660 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:36.780675 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:36.780798 (+ 123us) server_negotiation.cc:299] Negotiation successful
0929 19:10:36.780843 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
I20260929 19:10:36.781009 17563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:36.777856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:43681 (local address 127.0.0.1:51696)
0929 19:10:36.778001 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:36.778009 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:36.778059 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:36.778321 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:36.778322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:36.778325 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:36.778375 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:36.778377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:36.779248 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:36.779248 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:36.780124 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:36.780126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:36.780670 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:36.780671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:36.780739 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:36.780743 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:10:36.780789 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 19:10:36.780940 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":52,"threads_started":1}
I20260929 19:10:36.781065 17047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f0c0560533814a7cbebc5c5940f38a3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fde63ef43d8a440e96811f6d118232f2" is_pre_election: true
I20260929 19:10:36.781193 17047 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0c0560533814a7cbebc5c5940f38a3d in term 1.
I20260929 19:10:36.781268 16858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fde63ef43d8a440e96811f6d118232f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f0c0560533814a7cbebc5c5940f38a3d" is_pre_election: true
I20260929 19:10:36.781360 16836 leader_election.cc:302] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f0c0560533814a7cbebc5c5940f38a3d, fde63ef43d8a440e96811f6d118232f2; no voters: 64eaa0faf742494680dadbf96bb413f0
I20260929 19:10:36.781391 16858 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fde63ef43d8a440e96811f6d118232f2 in term 1.
I20260929 19:10:36.781458 17556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260929 19:10:36.781498 17556 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64eaa0faf742494680dadbf96bb413f0)
I20260929 19:10:36.781518 17556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 1 FOLLOWER]: Advancing to term 2
I20260929 19:10:36.781549 17021 leader_election.cc:302] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f0c0560533814a7cbebc5c5940f38a3d, fde63ef43d8a440e96811f6d118232f2; no voters: 64eaa0faf742494680dadbf96bb413f0
I20260929 19:10:36.781648 17558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260929 19:10:36.781682 17558 raft_consensus.cc:482] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 64eaa0faf742494680dadbf96bb413f0)
I20260929 19:10:36.781699 17558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 1 FOLLOWER]: Advancing to term 2
I20260929 19:10:36.782559 17558 raft_consensus.cc:504] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false } }
I20260929 19:10:36.782559 17556 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false } }
I20260929 19:10:36.782682 17556 leader_election.cc:288] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [CANDIDATE]: Term 2 election: Requested vote from peers 64eaa0faf742494680dadbf96bb413f0 (127.9.236.62:38041), fde63ef43d8a440e96811f6d118232f2 (127.9.236.60:41353)
I20260929 19:10:36.782703 17558 leader_election.cc:288] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [CANDIDATE]: Term 2 election: Requested vote from peers 64eaa0faf742494680dadbf96bb413f0 (127.9.236.62:38041), f0c0560533814a7cbebc5c5940f38a3d (127.9.236.61:43681)
I20260929 19:10:36.782977 17559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:36.782871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.0.0.1:34934)
0929 19:10:36.782915 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:36.782950 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260929 19:10:36.783088 16858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fde63ef43d8a440e96811f6d118232f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f0c0560533814a7cbebc5c5940f38a3d"
I20260929 19:10:36.783120 17563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:36.783013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.0.0.1:34942)
0929 19:10:36.783062 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:36.783095 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260929 19:10:36.783179 16858 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fde63ef43d8a440e96811f6d118232f2 in current term 2: Already voted for candidate f0c0560533814a7cbebc5c5940f38a3d in this term.
I20260929 19:10:36.783109 17047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f0c0560533814a7cbebc5c5940f38a3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fde63ef43d8a440e96811f6d118232f2"
I20260929 19:10:36.783223 17047 raft_consensus.cc:2379] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f0c0560533814a7cbebc5c5940f38a3d in current term 2: Already voted for candidate fde63ef43d8a440e96811f6d118232f2 in this term.
I20260929 19:10:36.783501 17559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:36.783407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.0.0.1:34944)
0929 19:10:36.783449 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:36.783477 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260929 19:10:36.783555 17563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:36.783457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.0.0.1:34952)
0929 19:10:36.783511 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:36.783537 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260929 19:10:36.783597 16836 leader_election.cc:334] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 64eaa0faf742494680dadbf96bb413f0 (127.9.236.62:38041): Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
I20260929 19:10:36.783653 16836 leader_election.cc:302] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0c0560533814a7cbebc5c5940f38a3d; no voters: 64eaa0faf742494680dadbf96bb413f0, fde63ef43d8a440e96811f6d118232f2
I20260929 19:10:36.783735 17556 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260929 19:10:36.783776 17024 leader_election.cc:334] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 64eaa0faf742494680dadbf96bb413f0 (127.9.236.62:38041): Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
I20260929 19:10:36.783815 17024 leader_election.cc:302] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fde63ef43d8a440e96811f6d118232f2; no voters: 64eaa0faf742494680dadbf96bb413f0, f0c0560533814a7cbebc5c5940f38a3d
I20260929 19:10:36.783901 17558 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260929 19:10:37.272671 17566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:37.272369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.1:45803)
0929 19:10:37.272566 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:37.272613 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":86,"threads_started":1}
I20260929 19:10:37.272727 17567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:37.272406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.3:35323)
0929 19:10:37.272606 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:37.272657 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":69,"threads_started":1}
I20260929 19:10:37.272776 17568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:37.272617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.2:33847)
0929 19:10:37.272707 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:37.272735 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":41,"threads_started":1}
I20260929 19:10:38.273419 17569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.273148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.1:37527)
0929 19:10:38.273322 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.273363 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":64,"threads_started":1}
I20260929 19:10:38.273555 17570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.273311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.2:51047)
0929 19:10:38.273442 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.273487 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260929 19:10:38.273566 17571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.273380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.3:48603)
0929 19:10:38.273489 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.273524 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":56,"threads_started":1}
I20260929 19:10:38.326367 17572 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:10:38.326483 17572 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false } }
I20260929 19:10:38.326659 17572 leader_election.cc:288] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 64eaa0faf742494680dadbf96bb413f0 (127.9.236.62:38041), fde63ef43d8a440e96811f6d118232f2 (127.9.236.60:41353)
I20260929 19:10:38.327132 17573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.326829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.0.0.1:35182)
0929 19:10:38.326994 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.327063 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":64,"threads_started":1}
I20260929 19:10:38.327135 17047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f0c0560533814a7cbebc5c5940f38a3d" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fde63ef43d8a440e96811f6d118232f2" is_pre_election: true
I20260929 19:10:38.327240 17047 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0c0560533814a7cbebc5c5940f38a3d in term 2.
I20260929 19:10:38.327566 16836 leader_election.cc:302] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0c0560533814a7cbebc5c5940f38a3d, fde63ef43d8a440e96811f6d118232f2; no voters:
I20260929 19:10:38.327620 17573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.327524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.0.0.1:35186)
0929 19:10:38.327566 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.327600 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260929 19:10:38.327674 17572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260929 19:10:38.327699 16836 leader_election.cc:334] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 64eaa0faf742494680dadbf96bb413f0 (127.9.236.62:38041): Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
I20260929 19:10:38.327736 17572 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:10:38.327767 17572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 2 FOLLOWER]: Advancing to term 3
I20260929 19:10:38.328527 17572 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false } }
I20260929 19:10:38.328634 17572 leader_election.cc:288] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [CANDIDATE]: Term 3 election: Requested vote from peers 64eaa0faf742494680dadbf96bb413f0 (127.9.236.62:38041), fde63ef43d8a440e96811f6d118232f2 (127.9.236.60:41353)
I20260929 19:10:38.328846 17573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.328722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.0.0.1:35192)
0929 19:10:38.328782 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.328818 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260929 19:10:38.328891 17047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f0c0560533814a7cbebc5c5940f38a3d" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fde63ef43d8a440e96811f6d118232f2"
I20260929 19:10:38.328958 17047 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 2 FOLLOWER]: Advancing to term 3
I20260929 19:10:38.329278 17573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.329212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.0.0.1:35198)
0929 19:10:38.329236 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.329260 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260929 19:10:38.329342 16836 leader_election.cc:334] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 64eaa0faf742494680dadbf96bb413f0 (127.9.236.62:38041): Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
I20260929 19:10:38.329699 17047 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0c0560533814a7cbebc5c5940f38a3d in term 3.
I20260929 19:10:38.329876 16836 leader_election.cc:302] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f0c0560533814a7cbebc5c5940f38a3d, fde63ef43d8a440e96811f6d118232f2; no voters: 64eaa0faf742494680dadbf96bb413f0
I20260929 19:10:38.329972 17572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 3 FOLLOWER]: Leader election won for term 3
I20260929 19:10:38.330127 17572 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [term 3 LEADER]: Becoming Leader. State: Replica: f0c0560533814a7cbebc5c5940f38a3d, State: Running, Role: LEADER
I20260929 19:10:38.330212 17572 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false } }
I20260929 19:10:38.330657 17575 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0c0560533814a7cbebc5c5940f38a3d. Latest consensus state: current_term: 3 leader_uuid: "f0c0560533814a7cbebc5c5940f38a3d" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false } } }
I20260929 19:10:38.330752 17575 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:38.330929 17577 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:38.331302 17577 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:10:38.331354 17577 catalog_manager.cc:1259] Loaded cluster ID: 5efb130dadec41a89dcc31b7161887e6
I20260929 19:10:38.331368 17577 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:10:38.331498 17577 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:38.331550 17577 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: Loaded TSK: 0
I20260929 19:10:38.331646 17577 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:10:38.383071 16848 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:10:38.385067 16848 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:10:38.386288 17047 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [term 3 FOLLOWER]: Refusing update from remote peer f0c0560533814a7cbebc5c5940f38a3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260929 19:10:38.386327 17573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.386211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.0.0.1:35220)
0929 19:10:38.386259 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.386298 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260929 19:10:38.386716 17573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.386582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.0.0.1:35222)
0929 19:10:38.386620 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.386669 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260929 19:10:38.386734 17575 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260929 19:10:38.386890 16836 consensus_peers.cc:487] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d -> Peer 64eaa0faf742494680dadbf96bb413f0 (127.9.236.62:38041): Couldn't send request to peer 64eaa0faf742494680dadbf96bb413f0. Status: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260929 19:10:38.387442 17579 sys_catalog.cc:455] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0c0560533814a7cbebc5c5940f38a3d. Latest consensus state: current_term: 3 leader_uuid: "f0c0560533814a7cbebc5c5940f38a3d" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false } } }
I20260929 19:10:38.387566 17579 sys_catalog.cc:458] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:10:38.387837 17578 mvcc.cc:205] Tried to move back new op lower bound from 7334744221227962368 to 7334744221001568256. Current Snapshot: MvccSnapshot[applied={T|T < 7334744221227962368}]
I20260929 19:10:38.387948 17572 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "f0c0560533814a7cbebc5c5940f38a3d" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false } } }
I20260929 19:10:38.388036 17572 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:38.388113 17580 mvcc.cc:205] Tried to move back new op lower bound from 7334744221227962368 to 7334744221001568256. Current Snapshot: MvccSnapshot[applied={T|T < 7334744221227962368}]
I20260929 19:10:38.388218 17579 sys_catalog.cc:455] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "f0c0560533814a7cbebc5c5940f38a3d" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "64eaa0faf742494680dadbf96bb413f0" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38041 } } peers { permanent_uuid: "f0c0560533814a7cbebc5c5940f38a3d" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 43681 } attrs { promote: false } } peers { permanent_uuid: "fde63ef43d8a440e96811f6d118232f2" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 41353 } attrs { promote: false } } }
I20260929 19:10:38.388309 17579 sys_catalog.cc:458] T 00000000000000000000000000000000 P fde63ef43d8a440e96811f6d118232f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:10:38.393486 17585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.390301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46578 (local address 127.9.236.2:39003)
0929 19:10:38.390577 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:38.390579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:38.390586 (+ 7us) server_negotiation.cc:407] Connection header received
0929 19:10:38.390607 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:38.390608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:38.390616 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:38.390841 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:38.390875 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:38.391547 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:38.392573 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:38.393140 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:38.393209 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:38.393230 (+ 21us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:38.393376 (+ 146us) server_negotiation.cc:299] Negotiation successful
0929 19:10:38.393413 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":74,"threads_started":1}
I20260929 19:10:38.393502 17573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.390170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:39003 (local address 127.0.0.1:46578)
0929 19:10:38.390205 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.390216 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:38.390264 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:38.390712 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:38.390712 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:38.390714 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:38.390773 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:38.390775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:38.391609 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:38.391610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:38.392484 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:38.392486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:38.393225 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:38.393226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:38.393302 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:38.393308 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 19:10:38.393366 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 19:10:38.393434 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260929 19:10:38.394191 17584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.390274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:46669 (local address 127.0.0.1:51608)
0929 19:10:38.390437 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.390446 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:38.390486 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:38.391173 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:38.391174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:38.391177 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:38.391234 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:38.391236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:38.392183 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:38.392184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:38.393334 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:38.393335 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:38.393946 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:38.393947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:38.394014 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:38.394017 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:10:38.394061 (+ 44us) client_negotiation.cc:240] Negotiation successful
0929 19:10:38.394108 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":46,"thread_start_us":101,"threads_started":1}
I20260929 19:10:38.394181 17587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.390836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51608 (local address 127.9.236.1:46669)
0929 19:10:38.390948 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:38.390950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:38.390959 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:38.390990 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:38.390991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:38.391003 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:38.391124 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:38.391297 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:38.392088 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:38.393413 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:38.393867 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:38.393938 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:38.393952 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:38.394071 (+ 119us) server_negotiation.cc:299] Negotiation successful
0929 19:10:38.394109 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":61,"threads_started":1}
I20260929 19:10:38.394413 17150 tablet_service.cc:1467] Processing CreateTablet for tablet 1b6dda5dbee842049add64964bb655fa (DEFAULT_TABLE table=test-table [id=29cc70a7807f462ab48fa44a42e3261c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:38.394534 17586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.390554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:43875 (local address 127.0.0.1:44836)
0929 19:10:38.390712 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.390723 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:38.390794 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:38.391247 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:38.391248 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:38.391251 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:38.391305 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:38.391307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:38.392218 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:38.392218 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:38.393469 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:38.393471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:38.394295 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:38.394296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:38.394376 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:38.394382 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 19:10:38.394408 (+ 26us) client_negotiation.cc:240] Negotiation successful
0929 19:10:38.394459 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":94,"threads_started":1}
I20260929 19:10:38.394550 17150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b6dda5dbee842049add64964bb655fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:38.394802 17287 tablet_service.cc:1467] Processing CreateTablet for tablet 1b6dda5dbee842049add64964bb655fa (DEFAULT_TABLE table=test-table [id=29cc70a7807f462ab48fa44a42e3261c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:38.394869 17588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.390851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44836 (local address 127.9.236.3:43875)
0929 19:10:38.391014 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:38.391017 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:38.391024 (+ 7us) server_negotiation.cc:407] Connection header received
0929 19:10:38.391055 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:38.391056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:38.391066 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:38.391147 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:38.391411 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:38.392131 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:38.393570 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:38.394197 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:38.394723 (+ 526us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:38.394739 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:38.394757 (+ 18us) server_negotiation.cc:299] Negotiation successful
0929 19:10:38.394802 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":103,"threads_started":1}
I20260929 19:10:38.394933 17287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b6dda5dbee842049add64964bb655fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:38.395831 17589 tablet_bootstrap.cc:492] T 1b6dda5dbee842049add64964bb655fa P 56e2d743ca3e4d018b7e997ec34773c0: Bootstrap starting.
I20260929 19:10:38.396138 17590 tablet_bootstrap.cc:492] T 1b6dda5dbee842049add64964bb655fa P 85ed2357da7247fc89f778b65e83aa92: Bootstrap starting.
I20260929 19:10:38.396301 17589 tablet_bootstrap.cc:654] T 1b6dda5dbee842049add64964bb655fa P 56e2d743ca3e4d018b7e997ec34773c0: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:38.396283 17423 tablet_service.cc:1467] Processing CreateTablet for tablet 1b6dda5dbee842049add64964bb655fa (DEFAULT_TABLE table=test-table [id=29cc70a7807f462ab48fa44a42e3261c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:38.396382 17423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b6dda5dbee842049add64964bb655fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:38.396546 17590 tablet_bootstrap.cc:654] T 1b6dda5dbee842049add64964bb655fa P 85ed2357da7247fc89f778b65e83aa92: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:38.396919 17589 tablet_bootstrap.cc:492] T 1b6dda5dbee842049add64964bb655fa P 56e2d743ca3e4d018b7e997ec34773c0: No bootstrap required, opened a new log
I20260929 19:10:38.396998 17589 ts_tablet_manager.cc:1397] T 1b6dda5dbee842049add64964bb655fa P 56e2d743ca3e4d018b7e997ec34773c0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:38.397157 17589 raft_consensus.cc:348] T 1b6dda5dbee842049add64964bb655fa P 56e2d743ca3e4d018b7e997ec34773c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 } } peers { permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 } } peers { permanent_uuid: "4bff726d322940009260aeb6fc83fa15" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43875 } }
I20260929 19:10:38.397236 17589 raft_consensus.cc:374] T 1b6dda5dbee842049add64964bb655fa P 56e2d743ca3e4d018b7e997ec34773c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:38.397253 17589 raft_consensus.cc:729] T 1b6dda5dbee842049add64964bb655fa P 56e2d743ca3e4d018b7e997ec34773c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e2d743ca3e4d018b7e997ec34773c0, State: Initialized, Role: FOLLOWER
I20260929 19:10:38.397318 17589 consensus_queue.cc:260] T 1b6dda5dbee842049add64964bb655fa P 56e2d743ca3e4d018b7e997ec34773c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 } } peers { permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 } } peers { permanent_uuid: "4bff726d322940009260aeb6fc83fa15" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43875 } }
I20260929 19:10:38.397519 17589 ts_tablet_manager.cc:1428] T 1b6dda5dbee842049add64964bb655fa P 56e2d743ca3e4d018b7e997ec34773c0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 19:10:38.397909 17590 tablet_bootstrap.cc:492] T 1b6dda5dbee842049add64964bb655fa P 85ed2357da7247fc89f778b65e83aa92: No bootstrap required, opened a new log
I20260929 19:10:38.398006 17569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.397903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.1:51427)
0929 19:10:38.397950 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.397980 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260929 19:10:38.398010 17590 ts_tablet_manager.cc:1397] T 1b6dda5dbee842049add64964bb655fa P 85ed2357da7247fc89f778b65e83aa92: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:10:38.398268 17590 raft_consensus.cc:348] T 1b6dda5dbee842049add64964bb655fa P 85ed2357da7247fc89f778b65e83aa92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 } } peers { permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 } } peers { permanent_uuid: "4bff726d322940009260aeb6fc83fa15" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43875 } }
I20260929 19:10:38.398353 17590 raft_consensus.cc:374] T 1b6dda5dbee842049add64964bb655fa P 85ed2357da7247fc89f778b65e83aa92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:38.398372 17590 raft_consensus.cc:729] T 1b6dda5dbee842049add64964bb655fa P 85ed2357da7247fc89f778b65e83aa92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85ed2357da7247fc89f778b65e83aa92, State: Initialized, Role: FOLLOWER
I20260929 19:10:38.398427 17590 consensus_queue.cc:260] T 1b6dda5dbee842049add64964bb655fa P 85ed2357da7247fc89f778b65e83aa92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 } } peers { permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 } } peers { permanent_uuid: "4bff726d322940009260aeb6fc83fa15" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43875 } }
I20260929 19:10:38.398541 17569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.398439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.1:33629)
0929 19:10:38.398495 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.398518 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 19:10:38.398633 17590 ts_tablet_manager.cc:1428] T 1b6dda5dbee842049add64964bb655fa P 85ed2357da7247fc89f778b65e83aa92: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:38.398665 17218 heartbeater.cc:503] Master 127.9.236.61:43681 was elected leader, sending a full tablet report...
I20260929 19:10:38.397758 17594 tablet_bootstrap.cc:492] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15: Bootstrap starting.
I20260929 19:10:38.398978 17570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.398894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.2:48085)
0929 19:10:38.398930 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.398952 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260929 19:10:38.399210 17355 heartbeater.cc:503] Master 127.9.236.61:43681 was elected leader, sending a full tablet report...
I20260929 19:10:38.399297 17594 tablet_bootstrap.cc:654] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:38.399636 17570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.399532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.2:51931)
0929 19:10:38.399586 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.399608 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260929 19:10:38.399888 17594 tablet_bootstrap.cc:492] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15: No bootstrap required, opened a new log
I20260929 19:10:38.399946 17594 ts_tablet_manager.cc:1397] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:10:38.400091 17594 raft_consensus.cc:348] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 } } peers { permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 } } peers { permanent_uuid: "4bff726d322940009260aeb6fc83fa15" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43875 } }
I20260929 19:10:38.400151 17594 raft_consensus.cc:374] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:38.400168 17594 raft_consensus.cc:729] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bff726d322940009260aeb6fc83fa15, State: Initialized, Role: FOLLOWER
I20260929 19:10:38.400213 17594 consensus_queue.cc:260] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 } } peers { permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 } } peers { permanent_uuid: "4bff726d322940009260aeb6fc83fa15" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43875 } }
I20260929 19:10:38.400381 17594 ts_tablet_manager.cc:1428] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 19:10:38.400646 17571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.400539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.3:49479)
0929 19:10:38.400583 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.400619 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260929 19:10:38.400800 17491 heartbeater.cc:503] Master 127.9.236.61:43681 was elected leader, sending a full tablet report...
I20260929 19:10:38.401057 17571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.400961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.3:46831)
0929 19:10:38.401008 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.401033 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260929 19:10:38.550520 17219 tablet.cc:2369] T 1b6dda5dbee842049add64964bb655fa P 56e2d743ca3e4d018b7e997ec34773c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 19:10:38.595589 17492 tablet.cc:2369] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:10:38.691324 17597 raft_consensus.cc:482] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:10:38.691526 17597 raft_consensus.cc:504] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 } } peers { permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 } } peers { permanent_uuid: "4bff726d322940009260aeb6fc83fa15" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43875 } }
I20260929 19:10:38.691764 17597 leader_election.cc:288] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85ed2357da7247fc89f778b65e83aa92 (127.9.236.2:39003), 56e2d743ca3e4d018b7e997ec34773c0 (127.9.236.1:46669)
I20260929 19:10:38.692088 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b6dda5dbee842049add64964bb655fa" candidate_uuid: "4bff726d322940009260aeb6fc83fa15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85ed2357da7247fc89f778b65e83aa92" is_pre_election: true
I20260929 19:10:38.692091 17170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b6dda5dbee842049add64964bb655fa" candidate_uuid: "4bff726d322940009260aeb6fc83fa15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e2d743ca3e4d018b7e997ec34773c0" is_pre_election: true
I20260929 19:10:38.692214 17170 raft_consensus.cc:2454] T 1b6dda5dbee842049add64964bb655fa P 56e2d743ca3e4d018b7e997ec34773c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bff726d322940009260aeb6fc83fa15 in term 0.
I20260929 19:10:38.692211 17307 raft_consensus.cc:2454] T 1b6dda5dbee842049add64964bb655fa P 85ed2357da7247fc89f778b65e83aa92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bff726d322940009260aeb6fc83fa15 in term 0.
I20260929 19:10:38.692473 17378 leader_election.cc:302] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bff726d322940009260aeb6fc83fa15, 56e2d743ca3e4d018b7e997ec34773c0; no voters:
I20260929 19:10:38.692591 17597 raft_consensus.cc:2781] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:10:38.692626 17597 raft_consensus.cc:482] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:10:38.692649 17597 raft_consensus.cc:3037] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:38.693622 17597 raft_consensus.cc:504] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 } } peers { permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 } } peers { permanent_uuid: "4bff726d322940009260aeb6fc83fa15" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43875 } }
I20260929 19:10:38.693771 17597 leader_election.cc:288] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 [CANDIDATE]: Term 1 election: Requested vote from peers 85ed2357da7247fc89f778b65e83aa92 (127.9.236.2:39003), 56e2d743ca3e4d018b7e997ec34773c0 (127.9.236.1:46669)
I20260929 19:10:38.693951 17170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b6dda5dbee842049add64964bb655fa" candidate_uuid: "4bff726d322940009260aeb6fc83fa15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e2d743ca3e4d018b7e997ec34773c0"
I20260929 19:10:38.693967 17307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b6dda5dbee842049add64964bb655fa" candidate_uuid: "4bff726d322940009260aeb6fc83fa15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85ed2357da7247fc89f778b65e83aa92"
I20260929 19:10:38.694042 17170 raft_consensus.cc:3037] T 1b6dda5dbee842049add64964bb655fa P 56e2d743ca3e4d018b7e997ec34773c0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:38.694053 17307 raft_consensus.cc:3037] T 1b6dda5dbee842049add64964bb655fa P 85ed2357da7247fc89f778b65e83aa92 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:38.694711 17307 raft_consensus.cc:2454] T 1b6dda5dbee842049add64964bb655fa P 85ed2357da7247fc89f778b65e83aa92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bff726d322940009260aeb6fc83fa15 in term 1.
I20260929 19:10:38.694761 17170 raft_consensus.cc:2454] T 1b6dda5dbee842049add64964bb655fa P 56e2d743ca3e4d018b7e997ec34773c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bff726d322940009260aeb6fc83fa15 in term 1.
I20260929 19:10:38.694944 17379 leader_election.cc:302] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bff726d322940009260aeb6fc83fa15, 85ed2357da7247fc89f778b65e83aa92; no voters:
I20260929 19:10:38.695060 17597 raft_consensus.cc:2781] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:38.695287 17597 raft_consensus.cc:686] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 [term 1 LEADER]: Becoming Leader. State: Replica: 4bff726d322940009260aeb6fc83fa15, State: Running, Role: LEADER
I20260929 19:10:38.695345 17597 consensus_queue.cc:237] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 } } peers { permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 } } peers { permanent_uuid: "4bff726d322940009260aeb6fc83fa15" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43875 } }
I20260929 19:10:38.695847 17571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.695701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.3:40959)
0929 19:10:38.695755 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.695810 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260929 19:10:38.696029 16848 catalog_manager.cc:5495] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bff726d322940009260aeb6fc83fa15 (127.9.236.3). New cstate: current_term: 1 leader_uuid: "4bff726d322940009260aeb6fc83fa15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bff726d322940009260aeb6fc83fa15" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 43875 } health_report { overall_health: HEALTHY } } }
I20260929 19:10:38.747087 17588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.744816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44840 (local address 127.9.236.3:43875)
0929 19:10:38.744888 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:38.744890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:38.744900 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:10:38.744928 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:38.744928 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:38.744945 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:38.744994 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:38.745142 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:38.745664 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:38.746435 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:38.746648 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:38.746723 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:38.746854 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:38.746886 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:38.746935 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:38.746983 (+ 48us) server_negotiation.cc:299] Negotiation successful
0929 19:10:38.747011 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 19:10:38.748136 17307 raft_consensus.cc:1261] T 1b6dda5dbee842049add64964bb655fa P 85ed2357da7247fc89f778b65e83aa92 [term 1 FOLLOWER]: Refusing update from remote peer 4bff726d322940009260aeb6fc83fa15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:38.748188 17170 raft_consensus.cc:1261] T 1b6dda5dbee842049add64964bb655fa P 56e2d743ca3e4d018b7e997ec34773c0 [term 1 FOLLOWER]: Refusing update from remote peer 4bff726d322940009260aeb6fc83fa15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:38.748454 17597 consensus_queue.cc:1035] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e2d743ca3e4d018b7e997ec34773c0" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:38.748600 17598 consensus_queue.cc:1035] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85ed2357da7247fc89f778b65e83aa92" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:38.749274 17569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.749164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.1:39693)
0929 19:10:38.749207 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.749239 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260929 19:10:38.750274 17570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.749852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.2:52417)
0929 19:10:38.750209 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.750243 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339}
I20260929 19:10:38.750391 17571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.750268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.3:35053)
0929 19:10:38.750312 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.750358 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260929 19:10:38.750985 17569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.750792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.1:52815)
0929 19:10:38.750924 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.750955 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260929 19:10:38.751606 17571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.751517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.3:46251)
0929 19:10:38.751552 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.751580 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260929 19:10:38.752214 17570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.752093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38041 (local address 127.9.236.2:41849)
0929 19:10:38.752152 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.752187 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 19:10:38.753870 16848 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:38.753950 16848 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:38.755340 16848 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f0c0560533814a7cbebc5c5940f38a3d: Sending DeleteTablet for 3 replicas of tablet 1b6dda5dbee842049add64964bb655fa
I20260929 19:10:38.755637 17150 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b6dda5dbee842049add64964bb655fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:38 UTC) from {username='oryx', principal='oryx/127.9.236.61@KRBTEST.COM'} at 127.0.0.1:51608
I20260929 19:10:38.755721 17423 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b6dda5dbee842049add64964bb655fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:38 UTC) from {username='oryx', principal='oryx/127.9.236.61@KRBTEST.COM'} at 127.0.0.1:44836
I20260929 19:10:38.755875 17611 tablet_replica.cc:330] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15: stopping tablet replica
I20260929 19:10:38.755928 17611 raft_consensus.cc:2229] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:10:38.755951 17287 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b6dda5dbee842049add64964bb655fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:38 UTC) from {username='oryx', principal='oryx/127.9.236.61@KRBTEST.COM'} at 127.0.0.1:46578
I20260929 19:10:38.756063 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 16999
I20260929 19:10:38.755985 17611 raft_consensus.cc:2258] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:38.756837 17612 tablet_replica.cc:330] T 1b6dda5dbee842049add64964bb655fa P 85ed2357da7247fc89f778b65e83aa92: stopping tablet replica
I20260929 19:10:38.756920 17612 raft_consensus.cc:2229] T 1b6dda5dbee842049add64964bb655fa P 85ed2357da7247fc89f778b65e83aa92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:38.756942 17611 ts_tablet_manager.cc:1905] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:38.756980 17612 raft_consensus.cc:2258] T 1b6dda5dbee842049add64964bb655fa P 85ed2357da7247fc89f778b65e83aa92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:38.757311 17612 ts_tablet_manager.cc:1905] T 1b6dda5dbee842049add64964bb655fa P 85ed2357da7247fc89f778b65e83aa92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:38.758801 17611 ts_tablet_manager.cc:1918] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:10:38.758824 17612 ts_tablet_manager.cc:1918] T 1b6dda5dbee842049add64964bb655fa P 85ed2357da7247fc89f778b65e83aa92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:10:38.758869 17611 log.cc:1196] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/1b6dda5dbee842049add64964bb655fa
I20260929 19:10:38.758888 17612 log.cc:1196] T 1b6dda5dbee842049add64964bb655fa P 85ed2357da7247fc89f778b65e83aa92: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/1b6dda5dbee842049add64964bb655fa
I20260929 19:10:38.759140 17611 ts_tablet_manager.cc:1939] T 1b6dda5dbee842049add64964bb655fa P 4bff726d322940009260aeb6fc83fa15: Deleting consensus metadata
I20260929 19:10:38.759158 17612 ts_tablet_manager.cc:1939] T 1b6dda5dbee842049add64964bb655fa P 85ed2357da7247fc89f778b65e83aa92: Deleting consensus metadata
I20260929 19:10:38.759508 16835 catalog_manager.cc:4841] TS 4bff726d322940009260aeb6fc83fa15 (127.9.236.3:43875): tablet 1b6dda5dbee842049add64964bb655fa (table test-table [id=29cc70a7807f462ab48fa44a42e3261c]) successfully deleted
I20260929 19:10:38.759519 16836 catalog_manager.cc:4841] TS 85ed2357da7247fc89f778b65e83aa92 (127.9.236.2:39003): tablet 1b6dda5dbee842049add64964bb655fa (table test-table [id=29cc70a7807f462ab48fa44a42e3261c]) successfully deleted
I20260929 19:10:38.762748 17586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:38.762625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:46669 (local address 127.0.0.1:51618)
0929 19:10:38.762669 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:38.762700 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.1:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260929 19:10:38.762872 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 17224
W20260929 19:10:38.762907 16835 catalog_manager.cc:4584] TS 56e2d743ca3e4d018b7e997ec34773c0 (127.9.236.1:46669): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1b6dda5dbee842049add64964bb655fa: Network error: Client connection negotiation failed: client connection to 127.9.236.1:46669: connect: Connection refused (error 111)
I20260929 19:10:38.768491 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 17360
I20260929 19:10:38.774034 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 16715
I20260929 19:10:38.778971 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 16903
2026-09-29T19:10:38Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6390 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:10:38 dist-test-slave-wftq krb5kdc[17615](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:10:38 dist-test-slave-wftq krb5kdc[17615](info): set up 2 sockets
Sep 29 19:10:38 dist-test-slave-wftq krb5kdc[17615](info): commencing operation
krb5kdc: starting...
W20260929 19:10:39.737564 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:10:39 dist-test-slave-wftq krb5kdc[17615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:10:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:10:39Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.62.keytab.
Entry for principal oryx/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.62.keytab.
I20260929 19:10:39.857924 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:35325
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.62.keytab
--principal=oryx/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:41899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:35325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:39.940344 17628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:39.940577 17628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:39.940598 17628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:39.941818 17628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:39.941859 17628 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 19:10:39.941871 17628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:39.941882 17628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:39.941893 17628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:39.941908 17628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:39.943454 17628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:41899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:35325
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.62.keytab
--principal=oryx/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:35325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:39.943688 17628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:39.943917 17628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:39.948908 17633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:39.948935 17634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:39.949214 17628 server_base.cc:1034] running on GCE node
W20260929 19:10:39.948935 17636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:39.949575 17628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:39.949891 17628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:39.951059 17628 hybrid_clock.cc:648] HybridClock initialized: now 1790709039951043 us; error 39 us; skew 500 ppm
Sep 29 19:10:39 dist-test-slave-wftq krb5kdc[17615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709039, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:39.957306 17628 init.cc:377] Logged in from keytab as oryx/127.9.236.62@KRBTEST.COM (short username oryx)
I20260929 19:10:39.957757 17628 webserver.cc:460] Webserver started at http://127.9.236.62:37671/ using document root <none> and password file <none>
I20260929 19:10:39.957971 17628 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:39.958014 17628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:39.958105 17628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:39.959174 17628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "19f9285149614f4eb38115d95d52e7fd"
format_stamp: "Formatted at 2026-09-29 19:10:39 on dist-test-slave-wftq"
server_key: "befa75c3f45c442ef7f971be82b42819"
server_key_iv: "f6c127853f089900defde12f72b8500d"
server_key_version: "encryptionkey@0"
I20260929 19:10:39.959470 17628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "19f9285149614f4eb38115d95d52e7fd"
format_stamp: "Formatted at 2026-09-29 19:10:39 on dist-test-slave-wftq"
server_key: "befa75c3f45c442ef7f971be82b42819"
server_key_iv: "f6c127853f089900defde12f72b8500d"
server_key_version: "encryptionkey@0"
I20260929 19:10:39.960788 17628 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:10:39.961566 17643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:39.961759 17628 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 19:10:39.961860 17628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "19f9285149614f4eb38115d95d52e7fd"
format_stamp: "Formatted at 2026-09-29 19:10:39 on dist-test-slave-wftq"
server_key: "befa75c3f45c442ef7f971be82b42819"
server_key_iv: "f6c127853f089900defde12f72b8500d"
server_key_version: "encryptionkey@0"
I20260929 19:10:39.961948 17628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:39.979940 17628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:39.981892 17628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:39.982002 17628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:39.986644 17628 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:35325
I20260929 19:10:39.986809 17705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:35325 every 8 connection(s)
I20260929 19:10:39.987013 17628 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:10:39.987759 17706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:39.990341 17706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd: Bootstrap starting.
I20260929 19:10:39.990978 17706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:39.991259 17706 log.cc:824] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:39.991958 17706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd: No bootstrap required, opened a new log
I20260929 19:10:39.993333 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 17628
I20260929 19:10:39.993448 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:10:39.993647 10160 external_mini_cluster.cc:1428] Setting key 94d05fe9de766e04ddd35b94a89e0233
I20260929 19:10:39.994431 17706 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19f9285149614f4eb38115d95d52e7fd" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35325 } }
I20260929 19:10:39.994596 17706 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:39.994619 17706 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19f9285149614f4eb38115d95d52e7fd, State: Initialized, Role: FOLLOWER
I20260929 19:10:39.994735 17706 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19f9285149614f4eb38115d95d52e7fd" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35325 } }
I20260929 19:10:39.994807 17706 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:10:39.994829 17706 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:10:39.994863 17706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:39.995571 17706 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19f9285149614f4eb38115d95d52e7fd" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35325 } }
I20260929 19:10:39.995687 17706 leader_election.cc:302] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19f9285149614f4eb38115d95d52e7fd; no voters:
I20260929 19:10:39.995847 17706 leader_election.cc:288] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:10:39.995899 17711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:39.996042 17711 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd [term 1 LEADER]: Becoming Leader. State: Replica: 19f9285149614f4eb38115d95d52e7fd, State: Running, Role: LEADER
I20260929 19:10:39.996121 17706 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:39.996124 17711 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19f9285149614f4eb38115d95d52e7fd" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35325 } }
I20260929 19:10:39.996503 17711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19f9285149614f4eb38115d95d52e7fd. Latest consensus state: current_term: 1 leader_uuid: "19f9285149614f4eb38115d95d52e7fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19f9285149614f4eb38115d95d52e7fd" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35325 } } }
I20260929 19:10:39.996578 17711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:39.996743 17713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19f9285149614f4eb38115d95d52e7fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19f9285149614f4eb38115d95d52e7fd" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35325 } } }
I20260929 19:10:39.996810 17713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:39.996929 17716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:39.997780 17716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:10:39.999601 17716 catalog_manager.cc:1347] Generated new cluster ID: 29c20af0409b4666a631ffbdbc77c69c
I20260929 19:10:39.999653 17716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:10:39 dist-test-slave-wftq krb5kdc[17615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM
I20260929 19:10:40.005882 17710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:39.994258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47110 (local address 127.9.236.62:35325)
0929 19:10:39.994486 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:39.994495 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:39.994525 (+ 30us) server_negotiation.cc:407] Connection header received
0929 19:10:39.994674 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:39.994681 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:39.994746 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:39.994897 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:39.995861 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:39.997304 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:39.998411 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:39.998995 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:39.999116 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.002994 (+ 3878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:40.002999 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:40.002999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:40.003013 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:40.004496 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:40.004796 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:40.004799 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:40.004800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:40.004885 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:40.005048 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:40.005049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:40.005049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:40.005323 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:40.005487 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:40.005551 (+ 64us) server_negotiation.cc:299] Negotiation successful
0929 19:10:40.005701 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":109,"threads_started":1}
I20260929 19:10:40.013227 17716 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:10:40.013792 17716 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:40.017158 17716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd: Generated new TSK 0
I20260929 19:10:40.017321 17716 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.236.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.236.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.236.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.1.keytab.
Entry for principal oryx/127.9.236.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.1.keytab.
I20260929 19:10:40.051576 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:35325
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.1.keytab
--principal=oryx/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:41899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:40.157780 17732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:40.157992 17732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:40.158023 17732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:40.159215 17732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:40.159255 17732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:40.159281 17732 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 19:10:40.159303 17732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:10:40.160774 17732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:41899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.1.keytab
--principal=oryx/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.236.62:35325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:40.161001 17732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:40.161250 17732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:40.166646 17737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:40.166712 17738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:40.166712 17740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:40.167232 17732 server_base.cc:1034] running on GCE node
I20260929 19:10:40.167466 17732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:40.167716 17732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:40.168866 17732 hybrid_clock.cc:648] HybridClock initialized: now 1790709040168830 us; error 58 us; skew 500 ppm
Sep 29 19:10:40 dist-test-slave-wftq krb5kdc[17615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709040, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:40.174180 17732 init.cc:377] Logged in from keytab as oryx/127.9.236.1@KRBTEST.COM (short username oryx)
I20260929 19:10:40.174639 17732 webserver.cc:460] Webserver started at http://127.9.236.1:37931/ using document root <none> and password file <none>
I20260929 19:10:40.174823 17732 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:40.174865 17732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:40.174961 17732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:40.175969 17732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "c1dc1c7cadc94204a8345df0f9140d83"
format_stamp: "Formatted at 2026-09-29 19:10:40 on dist-test-slave-wftq"
server_key: "81af2bd2269275597c8f103a1b11dd9a"
server_key_iv: "cb8cb840383b1da8353b5c6f7f0cf3a5"
server_key_version: "encryptionkey@0"
I20260929 19:10:40.176272 17732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "c1dc1c7cadc94204a8345df0f9140d83"
format_stamp: "Formatted at 2026-09-29 19:10:40 on dist-test-slave-wftq"
server_key: "81af2bd2269275597c8f103a1b11dd9a"
server_key_iv: "cb8cb840383b1da8353b5c6f7f0cf3a5"
server_key_version: "encryptionkey@0"
I20260929 19:10:40.177659 17732 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:10:40.178478 17747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:40.178691 17732 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:40.178777 17732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "c1dc1c7cadc94204a8345df0f9140d83"
format_stamp: "Formatted at 2026-09-29 19:10:40 on dist-test-slave-wftq"
server_key: "81af2bd2269275597c8f103a1b11dd9a"
server_key_iv: "cb8cb840383b1da8353b5c6f7f0cf3a5"
server_key_version: "encryptionkey@0"
I20260929 19:10:40.178854 17732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:40.202186 17732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:40.204389 17732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:40.204506 17732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:40.205140 17732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:40.205193 17732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:40.205224 17732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:40.205238 17732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 19:10:40 dist-test-slave-wftq krb5kdc[17615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709040, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM
I20260929 19:10:40.212689 17732 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:36447
I20260929 19:10:40.212844 17862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:36447 every 8 connection(s)
I20260929 19:10:40.213166 17732 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:10:40.215557 17710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.205659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:47453 (local address 127.9.236.62:35325)
0929 19:10:40.205737 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:40.205740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:40.206005 (+ 265us) server_negotiation.cc:407] Connection header received
0929 19:10:40.206739 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:40.206741 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:40.206759 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:40.206854 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:40.207625 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.208389 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.209588 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.209826 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.209889 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.212439 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:40.212441 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:40.212442 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:40.212454 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:40.214242 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:40.214612 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:40.214613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:40.214614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:40.214673 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:40.214886 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:40.214887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:40.214887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:40.215014 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:40.215071 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:40.215348 (+ 277us) server_negotiation.cc:299] Negotiation successful
0929 19:10:40.215459 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 19:10:40.215744 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.205626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:35325 (local address 127.9.236.1:47453)
0929 19:10:40.205843 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:40.205894 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:40.206623 (+ 729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:40.206975 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:40.206980 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:40.207275 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:40.207537 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.207542 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.208511 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.208512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.209491 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.209495 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.209925 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.209926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.210050 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.210739 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:40.210749 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:40.212328 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:40.214318 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:40.214324 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:40.214327 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:40.214546 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:40.214725 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:40.214726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:40.214726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:40.214832 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:40.215065 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:40.215071 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:40.215225 (+ 154us) client_negotiation.cc:769] Sending connection context
0929 19:10:40.215318 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 19:10:40.215509 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":114,"threads_started":1}
I20260929 19:10:40.216276 17863 heartbeater.cc:347] Connected to a master server at 127.9.236.62:35325
I20260929 19:10:40.216372 17863 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:40.216548 17863 heartbeater.cc:511] Master 127.9.236.62:35325 requested a full tablet report, sending...
I20260929 19:10:40.217217 17659 ts_manager.cc:194] Registered new tserver with Master: c1dc1c7cadc94204a8345df0f9140d83 (127.9.236.1:36447)
I20260929 19:10:40.218076 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 17732
I20260929 19:10:40.218189 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:10:40.218176 17659 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:47453
I20260929 19:10:40.218405 10160 external_mini_cluster.cc:1428] Setting key ab8501f80cb85f7356a53a10313bf7b0
I20260929 19:10:40.223495 17869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.217407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:35325 (local address 127.9.236.1:53669)
0929 19:10:40.217539 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:40.217551 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:40.217620 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:40.217746 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:40.217747 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:40.217949 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:40.218023 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.218025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.218927 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.218927 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.219746 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.219748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.220171 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.220172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.220226 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.220638 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:40.220641 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:40.220960 (+ 319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:40.222553 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:40.222554 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:40.222554 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:40.222708 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:40.222875 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:40.222875 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:40.222876 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:40.222933 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:40.223170 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:40.223171 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:40.223215 (+ 44us) client_negotiation.cc:769] Sending connection context
0929 19:10:40.223280 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 19:10:40.223390 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
I20260929 19:10:40.223490 17710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.217490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:53669 (local address 127.9.236.62:35325)
0929 19:10:40.217549 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:40.217551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:40.217609 (+ 58us) server_negotiation.cc:407] Connection header received
0929 19:10:40.217686 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:40.217687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:40.217695 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:40.217739 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:40.218095 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.218810 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.219828 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.220085 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.220177 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.221046 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:40.221048 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:40.221049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:40.221055 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:40.222474 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:40.222765 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:40.222766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:40.222766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:40.222818 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:40.222984 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:40.222985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:40.222985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:40.223118 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:40.223172 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:40.223307 (+ 135us) server_negotiation.cc:299] Negotiation successful
0929 19:10:40.223398 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
WARNING: no policy specified for oryx/127.9.236.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.236.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.236.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.2.keytab.
Entry for principal oryx/127.9.236.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.2.keytab.
I20260929 19:10:40.243408 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:35325
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.2.keytab
--principal=oryx/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:41899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:40.323807 17872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:40.324023 17872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:40.324045 17872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:40.325217 17872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:40.325258 17872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:40.325276 17872 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 19:10:40.325292 17872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:10:40.326756 17872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:41899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.2.keytab
--principal=oryx/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.236.62:35325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:40.326941 17872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:40.327118 17872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:40.332372 17880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:40.332372 17878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:40.332425 17877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:40.332696 17872 server_base.cc:1034] running on GCE node
I20260929 19:10:40.332959 17872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:40.333240 17872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:40.334411 17872 hybrid_clock.cc:648] HybridClock initialized: now 1790709040334388 us; error 40 us; skew 500 ppm
Sep 29 19:10:40 dist-test-slave-wftq krb5kdc[17615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709040, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:40.337260 17872 init.cc:377] Logged in from keytab as oryx/127.9.236.2@KRBTEST.COM (short username oryx)
I20260929 19:10:40.337749 17872 webserver.cc:460] Webserver started at http://127.9.236.2:41589/ using document root <none> and password file <none>
I20260929 19:10:40.337980 17872 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:40.338049 17872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:40.338187 17872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:40.339322 17872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f"
format_stamp: "Formatted at 2026-09-29 19:10:40 on dist-test-slave-wftq"
server_key: "4d436ac2e18bbd36171686319eda8a1b"
server_key_iv: "23f5fee5e262728691647c5be6d3f057"
server_key_version: "encryptionkey@0"
I20260929 19:10:40.339676 17872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f"
format_stamp: "Formatted at 2026-09-29 19:10:40 on dist-test-slave-wftq"
server_key: "4d436ac2e18bbd36171686319eda8a1b"
server_key_iv: "23f5fee5e262728691647c5be6d3f057"
server_key_version: "encryptionkey@0"
I20260929 19:10:40.341081 17872 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:10:40.341982 17887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:40.342207 17872 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:40.342303 17872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f"
format_stamp: "Formatted at 2026-09-29 19:10:40 on dist-test-slave-wftq"
server_key: "4d436ac2e18bbd36171686319eda8a1b"
server_key_iv: "23f5fee5e262728691647c5be6d3f057"
server_key_version: "encryptionkey@0"
I20260929 19:10:40.342373 17872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:40.356757 17872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:40.358980 17872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:40.359141 17872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:40.359767 17872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:40.359825 17872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:40.359862 17872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:40.359890 17872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 19:10:40 dist-test-slave-wftq krb5kdc[17615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709040, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM
I20260929 19:10:40.368667 17872 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:45039
I20260929 19:10:40.368783 18002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:45039 every 8 connection(s)
I20260929 19:10:40.369164 17872 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:10:40.370631 17710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.360357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:58101 (local address 127.9.236.62:35325)
0929 19:10:40.360416 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:40.360417 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:40.360713 (+ 296us) server_negotiation.cc:407] Connection header received
0929 19:10:40.361300 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:40.361302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:40.361314 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:40.361366 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:40.362219 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.362928 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.364350 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.364653 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.364720 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.367789 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:40.367792 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:40.367792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:40.367803 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:40.369328 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:40.369766 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:40.369768 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:40.369768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:40.369828 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:40.370038 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:40.370039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:40.370039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:40.370147 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:40.370204 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:40.370453 (+ 249us) server_negotiation.cc:299] Negotiation successful
0929 19:10:40.370529 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 19:10:40.370796 17896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.360338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:35325 (local address 127.9.236.2:58101)
0929 19:10:40.360557 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:40.360612 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:40.361205 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:40.361441 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:40.361444 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:40.361814 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:40.362115 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.362121 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.363007 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.363007 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.364271 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.364274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.364757 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.364759 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.364966 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.365649 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:40.365659 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:40.367638 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:40.369408 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:40.369414 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:40.369416 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:40.369706 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:40.369881 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:40.369882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:40.369882 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:40.369985 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:40.370199 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:40.370205 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:40.370338 (+ 133us) client_negotiation.cc:769] Sending connection context
0929 19:10:40.370425 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 19:10:40.370587 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":114,"threads_started":1}
I20260929 19:10:40.371315 18003 heartbeater.cc:347] Connected to a master server at 127.9.236.62:35325
I20260929 19:10:40.371446 18003 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:40.371651 18003 heartbeater.cc:511] Master 127.9.236.62:35325 requested a full tablet report, sending...
I20260929 19:10:40.372085 17660 ts_manager.cc:194] Registered new tserver with Master: 6ebaccd8c40746b28cdbbc2902fd6d0f (127.9.236.2:45039)
I20260929 19:10:40.372615 17660 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:58101
I20260929 19:10:40.377293 18009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.372278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:35325 (local address 127.9.236.2:55199)
0929 19:10:40.372406 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:40.372418 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:40.372488 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:40.372606 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:40.372607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:40.372822 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:40.372900 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.372902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.373559 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.373560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.374084 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.374085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.374454 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.374455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.374508 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.374845 (+ 337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:40.374848 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:40.375166 (+ 318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:40.376434 (+ 1268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:40.376435 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:40.376435 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:40.376595 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:40.376745 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:40.376745 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:40.376746 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:40.376789 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:40.376998 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:40.376999 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:40.377034 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 19:10:40.377088 (+ 54us) client_negotiation.cc:240] Negotiation successful
0929 19:10:40.377212 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":59,"threads_started":1}
I20260929 19:10:40.377303 17710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.372377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:55199 (local address 127.9.236.62:35325)
0929 19:10:40.372418 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:40.372420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:40.372478 (+ 58us) server_negotiation.cc:407] Connection header received
0929 19:10:40.372539 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:40.372539 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:40.372546 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:40.372597 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:40.372973 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.373466 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.374185 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.374376 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.374443 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.375245 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:40.375246 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:40.375246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:40.375252 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:40.376365 (+ 1113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:40.376647 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:40.376648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:40.376648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:40.376687 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:40.376827 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:40.376827 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:40.376828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:40.376946 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:40.376995 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:40.377181 (+ 186us) server_negotiation.cc:299] Negotiation successful
0929 19:10:40.377241 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:10:40.378746 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 17872
I20260929 19:10:40.378880 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:10:40.379103 10160 external_mini_cluster.cc:1428] Setting key 676940e8cba1971c3d3cac1bb4f0a031
WARNING: no policy specified for oryx/127.9.236.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.236.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.236.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.3.keytab.
Entry for principal oryx/127.9.236.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.3.keytab.
I20260929 19:10:40.404155 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:35325
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.3.keytab
--principal=oryx/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:41899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:40.483139 18012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:40.483340 18012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:40.483371 18012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:40.484498 18012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:40.484539 18012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:40.484565 18012 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 19:10:40.484586 18012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:10:40.486104 18012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:41899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.3.keytab
--principal=oryx/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.236.62:35325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:40.486323 18012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:40.486515 18012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:40.491618 18017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:40.491596 18020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:40.491583 18018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:40.491968 18012 server_base.cc:1034] running on GCE node
I20260929 19:10:40.492211 18012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:40.492436 18012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:40.493573 18012 hybrid_clock.cc:648] HybridClock initialized: now 1790709040493547 us; error 43 us; skew 500 ppm
Sep 29 19:10:40 dist-test-slave-wftq krb5kdc[17615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709040, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:40.495951 18012 init.cc:377] Logged in from keytab as oryx/127.9.236.3@KRBTEST.COM (short username oryx)
I20260929 19:10:40.496402 18012 webserver.cc:460] Webserver started at http://127.9.236.3:44533/ using document root <none> and password file <none>
I20260929 19:10:40.496616 18012 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:40.496670 18012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:40.496763 18012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:40.497866 18012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "17a50daf9a634b28a4a763ec9099c76a"
format_stamp: "Formatted at 2026-09-29 19:10:40 on dist-test-slave-wftq"
server_key: "155ea79b9f31b878f118c768a8245049"
server_key_iv: "76651dd0c888a71561ae2980d4676799"
server_key_version: "encryptionkey@0"
I20260929 19:10:40.498167 18012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "17a50daf9a634b28a4a763ec9099c76a"
format_stamp: "Formatted at 2026-09-29 19:10:40 on dist-test-slave-wftq"
server_key: "155ea79b9f31b878f118c768a8245049"
server_key_iv: "76651dd0c888a71561ae2980d4676799"
server_key_version: "encryptionkey@0"
I20260929 19:10:40.499461 18012 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 19:10:40.500168 18027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:40.500370 18012 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:40.500468 18012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "17a50daf9a634b28a4a763ec9099c76a"
format_stamp: "Formatted at 2026-09-29 19:10:40 on dist-test-slave-wftq"
server_key: "155ea79b9f31b878f118c768a8245049"
server_key_iv: "76651dd0c888a71561ae2980d4676799"
server_key_version: "encryptionkey@0"
I20260929 19:10:40.500548 18012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:40.517007 18012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:40.519325 18012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:40.519456 18012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:40.519999 18012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:40.520040 18012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:40.520063 18012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:40.520074 18012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 19:10:40 dist-test-slave-wftq krb5kdc[17615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709040, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM
I20260929 19:10:40.528146 18012 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:34375
I20260929 19:10:40.528292 18142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:34375 every 8 connection(s)
I20260929 19:10:40.528574 18012 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:10:40.529557 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 18012
I20260929 19:10:40.529668 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:10:40.529918 10160 external_mini_cluster.cc:1428] Setting key 3f748db1b51b9252db32ed42820e7a63
I20260929 19:10:40.530949 17710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.520518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:35913 (local address 127.9.236.62:35325)
0929 19:10:40.520598 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:40.520601 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:40.520790 (+ 189us) server_negotiation.cc:407] Connection header received
0929 19:10:40.521382 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:40.521384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:40.521398 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:40.521498 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:40.522320 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.523172 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.524686 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.524977 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.525052 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.527959 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:40.527960 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:40.527961 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:40.527968 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:40.529410 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:40.529907 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:40.529907 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:40.529908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:40.529963 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:40.530261 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:40.530261 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:40.530261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:40.530392 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:40.530459 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:40.530786 (+ 327us) server_negotiation.cc:299] Negotiation successful
0929 19:10:40.530881 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 19:10:40.531180 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.520467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:35325 (local address 127.9.236.3:35913)
0929 19:10:40.520652 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:40.520696 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:40.521272 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:40.521576 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:40.521581 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:40.521931 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:40.522211 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.522217 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.523271 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.523272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.524590 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.524594 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.525093 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.525094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.525329 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.525928 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:40.525940 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:40.527850 (+ 1910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:40.529492 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:40.529498 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:40.529502 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:40.529813 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:40.530076 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:40.530078 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:40.530078 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:40.530192 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:40.530463 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:40.530469 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:40.530636 (+ 167us) client_negotiation.cc:769] Sending connection context
0929 19:10:40.530760 (+ 124us) client_negotiation.cc:240] Negotiation successful
0929 19:10:40.530954 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
I20260929 19:10:40.531656 18143 heartbeater.cc:347] Connected to a master server at 127.9.236.62:35325
I20260929 19:10:40.531749 18143 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:40.531944 18143 heartbeater.cc:511] Master 127.9.236.62:35325 requested a full tablet report, sending...
I20260929 19:10:40.532342 17659 ts_manager.cc:194] Registered new tserver with Master: 17a50daf9a634b28a4a763ec9099c76a (127.9.236.3:34375)
I20260929 19:10:40.532824 17659 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:35913
I20260929 19:10:40.537823 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.532677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:35325 (local address 127.9.236.3:44807)
0929 19:10:40.532817 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:40.532830 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:40.532903 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:40.533043 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:40.533044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:40.533270 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:40.533350 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.533353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.534046 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.534047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.534557 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.534559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.534890 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.534891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.534937 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.535345 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:40.535349 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:40.535607 (+ 258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:40.536995 (+ 1388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:40.536996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:40.536996 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:40.537204 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:40.537359 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:40.537359 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:40.537359 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:40.537406 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:40.537582 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:40.537583 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:40.537616 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 19:10:40.537669 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 19:10:40.537747 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
I20260929 19:10:40.537813 17710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.532784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:44807 (local address 127.9.236.62:35325)
0929 19:10:40.532836 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:40.532838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:40.532916 (+ 78us) server_negotiation.cc:407] Connection header received
0929 19:10:40.532965 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:40.532966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:40.532976 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:40.533035 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:40.533432 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.533957 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.534627 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.534817 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.534881 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.535691 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:40.535693 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:40.535693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:40.535699 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:40.536928 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:40.537259 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:40.537260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:40.537260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:40.537306 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:40.537447 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:40.537447 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:40.537448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:40.537537 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:40.537583 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:40.537691 (+ 108us) server_negotiation.cc:299] Negotiation successful
0929 19:10:40.537749 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:10:40.541746 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 19:10:40 dist-test-slave-wftq krb5kdc[17615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709040, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 19:10:40 dist-test-slave-wftq krb5kdc[17615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:40 dist-test-slave-wftq krb5kdc[17615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
W20260929 19:10:40.563308 17710 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:40.557990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47118 (local address 127.9.236.62:35325)
0929 19:10:40.558079 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:40.558082 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:40.558095 (+ 13us) server_negotiation.cc:407] Connection header received
0929 19:10:40.558196 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:40.558197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:40.558206 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:40.558252 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:40.558819 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.559303 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.559984 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.560160 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.560212 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.563239 (+ 3027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47118: BlockingRecv error: recv got EOF from 127.0.0.1:47118 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 29 19:10:40 dist-test-slave-wftq krb5kdc[17615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709040, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM
I20260929 19:10:40.570564 17710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.564204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47130 (local address 127.9.236.62:35325)
0929 19:10:40.564251 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:40.564253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:40.564270 (+ 17us) server_negotiation.cc:407] Connection header received
0929 19:10:40.564330 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:40.564331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:40.564340 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:40.564391 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:40.564878 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.565465 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.566110 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.566301 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.566352 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.568305 (+ 1953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:40.568306 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:40.568307 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:40.568313 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:40.569610 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:40.569921 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:40.569922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:40.569923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:40.569980 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:40.570159 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:40.570159 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:40.570159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:40.570270 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:40.570354 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:40.570427 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 19:10:40.570492 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:10:40.572183 17659 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:10:40.574098 17659 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:10:40.580878 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.577134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40404 (local address 127.9.236.3:34375)
0929 19:10:40.577458 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:40.577462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:40.577471 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:40.577518 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:40.577520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:40.577535 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:40.577746 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:40.577803 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.578833 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.579926 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.580504 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.580572 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.580647 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:40.580739 (+ 92us) server_negotiation.cc:299] Negotiation successful
0929 19:10:40.580777 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":110,"threads_started":1}
I20260929 19:10:40.580853 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.576652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:36447 (local address 127.0.0.1:34902)
0929 19:10:40.576819 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:40.576861 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:40.576946 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:40.577397 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:40.577400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:40.577403 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:40.577486 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.577488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.578409 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.578410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.579600 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.579603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.580532 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.580533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.580622 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.580640 (+ 18us) client_negotiation.cc:769] Sending connection context
0929 19:10:40.580687 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 19:10:40.580756 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
I20260929 19:10:40.581069 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.576615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:45039 (local address 127.0.0.1:35980)
0929 19:10:40.576819 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:40.576863 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:40.576944 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:40.577450 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:40.577451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:40.577454 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:40.577524 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.577526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.578894 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.578895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.579965 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.579967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.580824 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.580825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.580913 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.580917 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:10:40.580947 (+ 30us) client_negotiation.cc:240] Negotiation successful
0929 19:10:40.581010 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":8,"thread_start_us":84,"threads_started":1}
I20260929 19:10:40.580904 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.576869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:34375 (local address 127.0.0.1:40404)
0929 19:10:40.577076 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:40.577084 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:40.577147 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:40.577638 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:40.577639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:40.577641 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:40.577678 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.577680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.578934 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.578935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.579831 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.579833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.580595 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.580596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.580663 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.580670 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 19:10:40.580721 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 19:10:40.580785 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":2,"thread_start_us":80,"threads_started":1}
I20260929 19:10:40.581378 18169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.576984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34902 (local address 127.9.236.1:36447)
0929 19:10:40.577212 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:40.577218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:40.577230 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:10:40.577273 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:40.577275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:40.577291 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:40.577383 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:40.577551 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.578332 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.579683 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.580428 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.581050 (+ 622us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.581167 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:40.581195 (+ 28us) server_negotiation.cc:299] Negotiation successful
0929 19:10:40.581272 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":106,"threads_started":1}
I20260929 19:10:40.581611 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.576741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35980 (local address 127.9.236.2:45039)
0929 19:10:40.577266 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:40.577271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:40.577281 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:10:40.577324 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:40.577326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:40.577342 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:40.577598 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:40.577653 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.578796 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.580042 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.580732 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.581360 (+ 628us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.581438 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:40.581464 (+ 26us) server_negotiation.cc:299] Negotiation successful
0929 19:10:40.581510 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":101,"threads_started":1}
I20260929 19:10:40.581993 18077 tablet_service.cc:1467] Processing CreateTablet for tablet 0352af039995473e82fe90c23dabd82b (DEFAULT_TABLE table=test-table [id=336221b5668a497695352bc13305eb62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:40.582070 17791 tablet_service.cc:1467] Processing CreateTablet for tablet 0352af039995473e82fe90c23dabd82b (DEFAULT_TABLE table=test-table [id=336221b5668a497695352bc13305eb62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:40.582398 18077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0352af039995473e82fe90c23dabd82b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:40.582472 17791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0352af039995473e82fe90c23dabd82b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:40.585422 18171 tablet_bootstrap.cc:492] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a: Bootstrap starting.
I20260929 19:10:40.585711 17937 tablet_service.cc:1467] Processing CreateTablet for tablet 0352af039995473e82fe90c23dabd82b (DEFAULT_TABLE table=test-table [id=336221b5668a497695352bc13305eb62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:40.586094 17937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0352af039995473e82fe90c23dabd82b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:40.586179 18171 tablet_bootstrap.cc:654] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:40.586520 18171 log.cc:824] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:40.587375 18171 tablet_bootstrap.cc:492] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a: No bootstrap required, opened a new log
I20260929 19:10:40.587456 18171 ts_tablet_manager.cc:1397] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:10:40.589668 18171 raft_consensus.cc:348] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } }
I20260929 19:10:40.589852 18171 raft_consensus.cc:374] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:40.589898 18171 raft_consensus.cc:729] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17a50daf9a634b28a4a763ec9099c76a, State: Initialized, Role: FOLLOWER
I20260929 19:10:40.590155 18173 tablet_bootstrap.cc:492] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83: Bootstrap starting.
I20260929 19:10:40.590194 18171 consensus_queue.cc:260] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } }
I20260929 19:10:40.590278 18174 tablet_bootstrap.cc:492] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f: Bootstrap starting.
I20260929 19:10:40.590503 18171 ts_tablet_manager.cc:1428] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:10:40.590680 18143 heartbeater.cc:503] Master 127.9.236.62:35325 was elected leader, sending a full tablet report...
I20260929 19:10:40.590848 18173 tablet_bootstrap.cc:654] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:40.590930 18174 tablet_bootstrap.cc:654] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:40.591169 18173 log.cc:824] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:40.591339 18174 log.cc:824] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:40.592028 18173 tablet_bootstrap.cc:492] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83: No bootstrap required, opened a new log
I20260929 19:10:40.592118 18173 ts_tablet_manager.cc:1397] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:10:40.592154 18174 tablet_bootstrap.cc:492] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f: No bootstrap required, opened a new log
I20260929 19:10:40.592219 18174 ts_tablet_manager.cc:1397] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:10:40.594223 18173 raft_consensus.cc:348] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } }
I20260929 19:10:40.594195 18174 raft_consensus.cc:348] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } }
I20260929 19:10:40.594359 18174 raft_consensus.cc:374] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:40.594359 18173 raft_consensus.cc:374] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:40.594393 18174 raft_consensus.cc:729] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ebaccd8c40746b28cdbbc2902fd6d0f, State: Initialized, Role: FOLLOWER
I20260929 19:10:40.594403 18173 raft_consensus.cc:729] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1dc1c7cadc94204a8345df0f9140d83, State: Initialized, Role: FOLLOWER
I20260929 19:10:40.594515 18174 consensus_queue.cc:260] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } }
I20260929 19:10:40.594515 18173 consensus_queue.cc:260] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } }
I20260929 19:10:40.594784 18174 ts_tablet_manager.cc:1428] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:10:40.594848 18003 heartbeater.cc:503] Master 127.9.236.62:35325 was elected leader, sending a full tablet report...
I20260929 19:10:40.594949 17863 heartbeater.cc:503] Master 127.9.236.62:35325 was elected leader, sending a full tablet report...
I20260929 19:10:40.594784 18173 ts_tablet_manager.cc:1428] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260929 19:10:40.619776 18004 tablet.cc:2369] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 19:10:40.714083 17864 tablet.cc:2369] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:10:40.764328 18179 raft_consensus.cc:482] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:10:40.764468 18179 raft_consensus.cc:504] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } }
I20260929 19:10:40.764832 18179 leader_election.cc:288] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1dc1c7cadc94204a8345df0f9140d83 (127.9.236.1:36447), 17a50daf9a634b28a4a763ec9099c76a (127.9.236.3:34375)
I20260929 19:10:40.768105 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.765432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:38627 (local address 127.9.236.3:34375)
0929 19:10:40.765481 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:40.765483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:40.765588 (+ 105us) server_negotiation.cc:407] Connection header received
0929 19:10:40.765651 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:40.765652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:40.765664 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:40.765714 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:40.765874 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.766391 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.767386 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.767756 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.767824 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.767834 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:40.767998 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 19:10:40.768032 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 19:10:40.768121 18169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.765157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:58287 (local address 127.9.236.1:36447)
0929 19:10:40.765228 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:40.765231 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:40.765293 (+ 62us) server_negotiation.cc:407] Connection header received
0929 19:10:40.765389 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:40.765391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:40.765404 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:40.765450 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:40.765682 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.766297 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.767395 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.767755 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.767820 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.767840 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:40.768001 (+ 161us) server_negotiation.cc:299] Negotiation successful
0929 19:10:40.768040 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 19:10:40.768244 17896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.765089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:36447 (local address 127.9.236.2:58287)
0929 19:10:40.765204 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:40.765242 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:40.765313 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:40.765507 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:40.765507 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:40.765509 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:40.765582 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.765584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.766392 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.766394 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.767301 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.767304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.767839 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.767841 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.767928 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.767931 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:10:40.767983 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 19:10:40.768038 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":50}
I20260929 19:10:40.768255 18180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.765258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:34375 (local address 127.9.236.2:38627)
0929 19:10:40.765505 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:40.765516 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:40.765587 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:40.765725 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:40.765726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:40.765729 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:40.765809 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.765811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.766470 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.766470 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.767296 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:40.767297 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.767834 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.767835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:40.767928 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.767932 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:10:40.767983 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 19:10:40.768030 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":145,"threads_started":1}
I20260929 19:10:40.768680 18097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0352af039995473e82fe90c23dabd82b" candidate_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17a50daf9a634b28a4a763ec9099c76a" is_pre_election: true
I20260929 19:10:40.768822 18097 raft_consensus.cc:2454] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ebaccd8c40746b28cdbbc2902fd6d0f in term 0.
I20260929 19:10:40.769029 17888 leader_election.cc:302] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17a50daf9a634b28a4a763ec9099c76a, 6ebaccd8c40746b28cdbbc2902fd6d0f; no voters:
I20260929 19:10:40.769021 17817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0352af039995473e82fe90c23dabd82b" candidate_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1dc1c7cadc94204a8345df0f9140d83" is_pre_election: true
I20260929 19:10:40.769171 18179 raft_consensus.cc:2781] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:10:40.769207 17817 raft_consensus.cc:2454] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ebaccd8c40746b28cdbbc2902fd6d0f in term 0.
I20260929 19:10:40.769227 18179 raft_consensus.cc:482] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:10:40.769250 18179 raft_consensus.cc:3037] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:40.770227 18179 raft_consensus.cc:504] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } }
I20260929 19:10:40.770349 18179 leader_election.cc:288] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f [CANDIDATE]: Term 1 election: Requested vote from peers c1dc1c7cadc94204a8345df0f9140d83 (127.9.236.1:36447), 17a50daf9a634b28a4a763ec9099c76a (127.9.236.3:34375)
I20260929 19:10:40.770628 17817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0352af039995473e82fe90c23dabd82b" candidate_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1dc1c7cadc94204a8345df0f9140d83"
I20260929 19:10:40.770632 18097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0352af039995473e82fe90c23dabd82b" candidate_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17a50daf9a634b28a4a763ec9099c76a"
I20260929 19:10:40.770714 18097 raft_consensus.cc:3037] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:40.770715 17817 raft_consensus.cc:3037] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:40.771409 18097 raft_consensus.cc:2454] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ebaccd8c40746b28cdbbc2902fd6d0f in term 1.
I20260929 19:10:40.771586 17817 raft_consensus.cc:2454] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ebaccd8c40746b28cdbbc2902fd6d0f in term 1.
I20260929 19:10:40.771584 17888 leader_election.cc:302] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17a50daf9a634b28a4a763ec9099c76a, 6ebaccd8c40746b28cdbbc2902fd6d0f; no voters:
I20260929 19:10:40.771718 18179 raft_consensus.cc:2781] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:40.771903 18179 raft_consensus.cc:686] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 LEADER]: Becoming Leader. State: Replica: 6ebaccd8c40746b28cdbbc2902fd6d0f, State: Running, Role: LEADER
I20260929 19:10:40.772015 18179 consensus_queue.cc:237] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } }
I20260929 19:10:40.772946 17660 catalog_manager.cc:5495] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ebaccd8c40746b28cdbbc2902fd6d0f (127.9.236.2). New cstate: current_term: 1 leader_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } health_report { overall_health: HEALTHY } } }
W20260929 19:10:40.779230 18144 tablet.cc:2369] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:10:40.806052 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:40.803602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35982 (local address 127.9.236.2:45039)
0929 19:10:40.803653 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:40.803655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:40.803681 (+ 26us) server_negotiation.cc:407] Connection header received
0929 19:10:40.803736 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:40.803737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:40.803766 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:40.803843 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:40.803971 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.804504 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.805338 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:40.805560 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:40.805619 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:40.805748 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:40.805876 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:40.805938 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:40.805961 (+ 23us) server_negotiation.cc:299] Negotiation successful
0929 19:10:40.805981 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:10:40.808058 18097 raft_consensus.cc:1261] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a [term 1 FOLLOWER]: Refusing update from remote peer 6ebaccd8c40746b28cdbbc2902fd6d0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:40.808061 17817 raft_consensus.cc:1261] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83 [term 1 FOLLOWER]: Refusing update from remote peer 6ebaccd8c40746b28cdbbc2902fd6d0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:40.808465 18181 consensus_queue.cc:1035] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:40.808604 18179 consensus_queue.cc:1035] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:40.810918 18184 mvcc.cc:205] Tried to move back new op lower bound from 7334744231146934272 to 7334744231002873856. Current Snapshot: MvccSnapshot[applied={T|T < 7334744231146934272}]
I20260929 19:10:40.811826 18186 mvcc.cc:205] Tried to move back new op lower bound from 7334744231146934272 to 7334744231002873856. Current Snapshot: MvccSnapshot[applied={T|T < 7334744231146934272}]
I20260929 19:10:40.814106 17658 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:40.814203 17658 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:40.814971 17658 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd: Sending DeleteTablet for 3 replicas of tablet 0352af039995473e82fe90c23dabd82b
I20260929 19:10:40.815372 17937 tablet_service.cc:1514] Processing DeleteTablet for tablet 0352af039995473e82fe90c23dabd82b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:40 UTC) from {username='oryx', principal='oryx/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:35980
I20260929 19:10:40.815387 17791 tablet_service.cc:1514] Processing DeleteTablet for tablet 0352af039995473e82fe90c23dabd82b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:40 UTC) from {username='oryx', principal='oryx/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:34902
I20260929 19:10:40.815598 18077 tablet_service.cc:1514] Processing DeleteTablet for tablet 0352af039995473e82fe90c23dabd82b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:40 UTC) from {username='oryx', principal='oryx/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:40404
I20260929 19:10:40.815616 17658 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:10:40.815752 17658 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:10:40.815834 18193 tablet_replica.cc:330] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f: stopping tablet replica
I20260929 19:10:40.815932 18193 raft_consensus.cc:2229] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:10:40.816109 18193 raft_consensus.cc:2258] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:40.816672 18193 ts_tablet_manager.cc:1905] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:40.816948 17937 tablet_service.cc:1467] Processing CreateTablet for tablet 54b3c484dcd4462cba4e4d9b39f85212 (DEFAULT_TABLE table=test-table [id=978e25abe77745dcaacb8ff9455e2889]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:40.817057 17937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54b3c484dcd4462cba4e4d9b39f85212. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:40.818105 17791 tablet_service.cc:1467] Processing CreateTablet for tablet 54b3c484dcd4462cba4e4d9b39f85212 (DEFAULT_TABLE table=test-table [id=978e25abe77745dcaacb8ff9455e2889]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:40.818233 17791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54b3c484dcd4462cba4e4d9b39f85212. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:40.818360 18077 tablet_service.cc:1467] Processing CreateTablet for tablet 54b3c484dcd4462cba4e4d9b39f85212 (DEFAULT_TABLE table=test-table [id=978e25abe77745dcaacb8ff9455e2889]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:40.818519 18077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54b3c484dcd4462cba4e4d9b39f85212. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:40.818540 18193 ts_tablet_manager.cc:1918] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:10:40.818612 18193 log.cc:1196] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/0352af039995473e82fe90c23dabd82b
I20260929 19:10:40.818854 18193 ts_tablet_manager.cc:1939] T 0352af039995473e82fe90c23dabd82b P 6ebaccd8c40746b28cdbbc2902fd6d0f: Deleting consensus metadata
I20260929 19:10:40.819216 17645 catalog_manager.cc:4841] TS 6ebaccd8c40746b28cdbbc2902fd6d0f (127.9.236.2:45039): tablet 0352af039995473e82fe90c23dabd82b (table test-table [id=336221b5668a497695352bc13305eb62]) successfully deleted
I20260929 19:10:40.820138 18174 tablet_bootstrap.cc:492] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Bootstrap starting.
I20260929 19:10:40.820390 18173 tablet_bootstrap.cc:492] T 54b3c484dcd4462cba4e4d9b39f85212 P c1dc1c7cadc94204a8345df0f9140d83: Bootstrap starting.
I20260929 19:10:40.820628 18174 tablet_bootstrap.cc:654] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:40.820641 18194 tablet_replica.cc:330] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83: stopping tablet replica
I20260929 19:10:40.820744 18194 raft_consensus.cc:2229] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:40.820781 18173 tablet_bootstrap.cc:654] T 54b3c484dcd4462cba4e4d9b39f85212 P c1dc1c7cadc94204a8345df0f9140d83: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:40.820844 18194 raft_consensus.cc:2258] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:40.820976 18171 tablet_bootstrap.cc:492] T 54b3c484dcd4462cba4e4d9b39f85212 P 17a50daf9a634b28a4a763ec9099c76a: Bootstrap starting.
I20260929 19:10:40.821010 18195 tablet_replica.cc:330] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a: stopping tablet replica
I20260929 19:10:40.821148 18195 raft_consensus.cc:2229] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:40.821255 18195 raft_consensus.cc:2258] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:40.821480 18171 tablet_bootstrap.cc:654] T 54b3c484dcd4462cba4e4d9b39f85212 P 17a50daf9a634b28a4a763ec9099c76a: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:40.821555 18174 tablet_bootstrap.cc:492] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f: No bootstrap required, opened a new log
I20260929 19:10:40.821614 18174 ts_tablet_manager.cc:1397] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:40.821777 18194 ts_tablet_manager.cc:1905] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:40.821767 18174 raft_consensus.cc:348] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:40.821841 18174 raft_consensus.cc:374] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:40.821859 18174 raft_consensus.cc:729] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ebaccd8c40746b28cdbbc2902fd6d0f, State: Initialized, Role: FOLLOWER
I20260929 19:10:40.821902 18174 consensus_queue.cc:260] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:40.821987 18174 ts_tablet_manager.cc:1428] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 19:10:40.822374 18195 ts_tablet_manager.cc:1905] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:40.823830 18195 ts_tablet_manager.cc:1918] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:10:40.823905 18195 log.cc:1196] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/0352af039995473e82fe90c23dabd82b
I20260929 19:10:40.824299 18195 ts_tablet_manager.cc:1939] T 0352af039995473e82fe90c23dabd82b P 17a50daf9a634b28a4a763ec9099c76a: Deleting consensus metadata
I20260929 19:10:40.824738 17644 catalog_manager.cc:4841] TS 17a50daf9a634b28a4a763ec9099c76a (127.9.236.3:34375): tablet 0352af039995473e82fe90c23dabd82b (table test-table [id=336221b5668a497695352bc13305eb62]) successfully deleted
I20260929 19:10:40.824976 18194 ts_tablet_manager.cc:1918] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:10:40.825031 18194 log.cc:1196] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-0/wal/wals/0352af039995473e82fe90c23dabd82b
I20260929 19:10:40.825240 18194 ts_tablet_manager.cc:1939] T 0352af039995473e82fe90c23dabd82b P c1dc1c7cadc94204a8345df0f9140d83: Deleting consensus metadata
I20260929 19:10:40.825683 18173 tablet_bootstrap.cc:492] T 54b3c484dcd4462cba4e4d9b39f85212 P c1dc1c7cadc94204a8345df0f9140d83: No bootstrap required, opened a new log
I20260929 19:10:40.825723 17644 catalog_manager.cc:4841] TS c1dc1c7cadc94204a8345df0f9140d83 (127.9.236.1:36447): tablet 0352af039995473e82fe90c23dabd82b (table test-table [id=336221b5668a497695352bc13305eb62]) successfully deleted
I20260929 19:10:40.825747 18173 ts_tablet_manager.cc:1397] T 54b3c484dcd4462cba4e4d9b39f85212 P c1dc1c7cadc94204a8345df0f9140d83: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260929 19:10:40.825937 18173 raft_consensus.cc:348] T 54b3c484dcd4462cba4e4d9b39f85212 P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:40.826035 18173 raft_consensus.cc:374] T 54b3c484dcd4462cba4e4d9b39f85212 P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:40.826081 18173 raft_consensus.cc:729] T 54b3c484dcd4462cba4e4d9b39f85212 P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1dc1c7cadc94204a8345df0f9140d83, State: Initialized, Role: FOLLOWER
I20260929 19:10:40.826140 18173 consensus_queue.cc:260] T 54b3c484dcd4462cba4e4d9b39f85212 P c1dc1c7cadc94204a8345df0f9140d83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:40.826278 18173 ts_tablet_manager.cc:1428] T 54b3c484dcd4462cba4e4d9b39f85212 P c1dc1c7cadc94204a8345df0f9140d83: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 19:10:40.832268 18171 tablet_bootstrap.cc:492] T 54b3c484dcd4462cba4e4d9b39f85212 P 17a50daf9a634b28a4a763ec9099c76a: No bootstrap required, opened a new log
I20260929 19:10:40.832336 18171 ts_tablet_manager.cc:1397] T 54b3c484dcd4462cba4e4d9b39f85212 P 17a50daf9a634b28a4a763ec9099c76a: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20260929 19:10:40.832521 18171 raft_consensus.cc:348] T 54b3c484dcd4462cba4e4d9b39f85212 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:40.832608 18171 raft_consensus.cc:374] T 54b3c484dcd4462cba4e4d9b39f85212 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:40.832628 18171 raft_consensus.cc:729] T 54b3c484dcd4462cba4e4d9b39f85212 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17a50daf9a634b28a4a763ec9099c76a, State: Initialized, Role: FOLLOWER
I20260929 19:10:40.832701 18171 consensus_queue.cc:260] T 54b3c484dcd4462cba4e4d9b39f85212 P 17a50daf9a634b28a4a763ec9099c76a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:40.832801 18171 ts_tablet_manager.cc:1428] T 54b3c484dcd4462cba4e4d9b39f85212 P 17a50daf9a634b28a4a763ec9099c76a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260929 19:10:41.007762 18181 raft_consensus.cc:482] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:10:41.007866 18181 raft_consensus.cc:504] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.008031 18181 leader_election.cc:288] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1dc1c7cadc94204a8345df0f9140d83 (127.9.236.1:36447), 17a50daf9a634b28a4a763ec9099c76a (127.9.236.3:34375)
I20260929 19:10:41.008322 17817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54b3c484dcd4462cba4e4d9b39f85212" candidate_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1dc1c7cadc94204a8345df0f9140d83" is_pre_election: true
I20260929 19:10:41.008425 17817 raft_consensus.cc:2454] T 54b3c484dcd4462cba4e4d9b39f85212 P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ebaccd8c40746b28cdbbc2902fd6d0f in term 0.
I20260929 19:10:41.008355 18097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54b3c484dcd4462cba4e4d9b39f85212" candidate_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17a50daf9a634b28a4a763ec9099c76a" is_pre_election: true
I20260929 19:10:41.008519 18097 raft_consensus.cc:2454] T 54b3c484dcd4462cba4e4d9b39f85212 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ebaccd8c40746b28cdbbc2902fd6d0f in term 0.
I20260929 19:10:41.008584 17888 leader_election.cc:302] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ebaccd8c40746b28cdbbc2902fd6d0f, c1dc1c7cadc94204a8345df0f9140d83; no voters:
I20260929 19:10:41.008724 18181 raft_consensus.cc:2781] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:10:41.008777 18181 raft_consensus.cc:482] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:10:41.008797 18181 raft_consensus.cc:3037] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:41.009750 18181 raft_consensus.cc:504] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.009902 18181 leader_election.cc:288] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f [CANDIDATE]: Term 1 election: Requested vote from peers c1dc1c7cadc94204a8345df0f9140d83 (127.9.236.1:36447), 17a50daf9a634b28a4a763ec9099c76a (127.9.236.3:34375)
I20260929 19:10:41.010113 18097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54b3c484dcd4462cba4e4d9b39f85212" candidate_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17a50daf9a634b28a4a763ec9099c76a"
I20260929 19:10:41.010114 17817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54b3c484dcd4462cba4e4d9b39f85212" candidate_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1dc1c7cadc94204a8345df0f9140d83"
I20260929 19:10:41.010196 18097 raft_consensus.cc:3037] T 54b3c484dcd4462cba4e4d9b39f85212 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:41.010197 17817 raft_consensus.cc:3037] T 54b3c484dcd4462cba4e4d9b39f85212 P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:41.011060 18097 raft_consensus.cc:2454] T 54b3c484dcd4462cba4e4d9b39f85212 P 17a50daf9a634b28a4a763ec9099c76a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ebaccd8c40746b28cdbbc2902fd6d0f in term 1.
I20260929 19:10:41.011274 17888 leader_election.cc:302] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17a50daf9a634b28a4a763ec9099c76a, 6ebaccd8c40746b28cdbbc2902fd6d0f; no voters:
I20260929 19:10:41.011394 18181 raft_consensus.cc:2781] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:41.011456 18181 raft_consensus.cc:686] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 LEADER]: Becoming Leader. State: Replica: 6ebaccd8c40746b28cdbbc2902fd6d0f, State: Running, Role: LEADER
I20260929 19:10:41.011451 17817 raft_consensus.cc:2454] T 54b3c484dcd4462cba4e4d9b39f85212 P c1dc1c7cadc94204a8345df0f9140d83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ebaccd8c40746b28cdbbc2902fd6d0f in term 1.
I20260929 19:10:41.011518 18181 consensus_queue.cc:237] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.012290 17658 catalog_manager.cc:5495] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ebaccd8c40746b28cdbbc2902fd6d0f (127.9.236.2). New cstate: current_term: 1 leader_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } health_report { overall_health: UNKNOWN } } }
Sep 29 19:10:41 dist-test-slave-wftq krb5kdc[17615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709039, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.62@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM
I20260929 19:10:41.081168 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:41.074001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:35325 (local address 127.0.0.1:47134)
0929 19:10:41.074178 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:41.074188 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:41.074260 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:41.074434 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:41.074435 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:41.074770 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:41.074823 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:41.074825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.075506 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.075507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:41.076018 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:41.076019 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.076326 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.076327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:41.076379 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:41.076933 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:41.076938 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:41.078530 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:41.080107 (+ 1577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:41.080113 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:41.080116 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:41.080313 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:41.080521 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:41.080522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:41.080522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:41.080589 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:41.080820 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:41.080826 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:41.080878 (+ 52us) client_negotiation.cc:769] Sending connection context
0929 19:10:41.080931 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 19:10:41.081034 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":92,"threads_started":1}
I20260929 19:10:41.081290 18206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:41.074075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47134 (local address 127.9.236.62:35325)
0929 19:10:41.074274 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:41.074276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:41.074287 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:10:41.074315 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:41.074316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:41.074329 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:41.074397 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:41.074908 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.075442 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.076088 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.076272 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.076325 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:41.078647 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:41.078648 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:41.078649 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:41.078658 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:41.080016 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:41.080379 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:41.080380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:41.080380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:41.080434 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:41.080652 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:41.080652 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:41.080652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:41.080767 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:41.080817 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:41.080981 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 19:10:41.081093 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":178,"thread_start_us":129,"threads_started":1}
I20260929 19:10:41.083439 17658 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:47134:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 19:10:41.083655 17658 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:10:41.085270 17791 tablet_service.cc:1467] Processing CreateTablet for tablet 5f9d5c1b802741cca3eb860e859913c7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a4eb39afc4ed4d258e23ac8a11a1c88f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 19:10:41.085294 18077 tablet_service.cc:1467] Processing CreateTablet for tablet 5f9d5c1b802741cca3eb860e859913c7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a4eb39afc4ed4d258e23ac8a11a1c88f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 19:10:41.085418 17791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f9d5c1b802741cca3eb860e859913c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:41.085464 18077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f9d5c1b802741cca3eb860e859913c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:41.086723 18171 tablet_bootstrap.cc:492] T 5f9d5c1b802741cca3eb860e859913c7 P 17a50daf9a634b28a4a763ec9099c76a: Bootstrap starting.
I20260929 19:10:41.086815 17937 tablet_service.cc:1467] Processing CreateTablet for tablet 5f9d5c1b802741cca3eb860e859913c7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a4eb39afc4ed4d258e23ac8a11a1c88f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 19:10:41.086974 17937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f9d5c1b802741cca3eb860e859913c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:41.087175 18171 tablet_bootstrap.cc:654] T 5f9d5c1b802741cca3eb860e859913c7 P 17a50daf9a634b28a4a763ec9099c76a: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:41.087782 18171 tablet_bootstrap.cc:492] T 5f9d5c1b802741cca3eb860e859913c7 P 17a50daf9a634b28a4a763ec9099c76a: No bootstrap required, opened a new log
I20260929 19:10:41.087826 18171 ts_tablet_manager.cc:1397] T 5f9d5c1b802741cca3eb860e859913c7 P 17a50daf9a634b28a4a763ec9099c76a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:41.087965 18171 raft_consensus.cc:348] T 5f9d5c1b802741cca3eb860e859913c7 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.088038 18171 raft_consensus.cc:374] T 5f9d5c1b802741cca3eb860e859913c7 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:41.088052 18171 raft_consensus.cc:729] T 5f9d5c1b802741cca3eb860e859913c7 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17a50daf9a634b28a4a763ec9099c76a, State: Initialized, Role: FOLLOWER
I20260929 19:10:41.088096 18171 consensus_queue.cc:260] T 5f9d5c1b802741cca3eb860e859913c7 P 17a50daf9a634b28a4a763ec9099c76a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.088191 18171 ts_tablet_manager.cc:1428] T 5f9d5c1b802741cca3eb860e859913c7 P 17a50daf9a634b28a4a763ec9099c76a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260929 19:10:41.088222 18173 tablet_bootstrap.cc:492] T 5f9d5c1b802741cca3eb860e859913c7 P c1dc1c7cadc94204a8345df0f9140d83: Bootstrap starting.
I20260929 19:10:41.088322 18175 tablet_replica.cc:439] T 5f9d5c1b802741cca3eb860e859913c7 P 17a50daf9a634b28a4a763ec9099c76a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } } }
I20260929 19:10:41.088399 18175 tablet_replica.cc:442] T 5f9d5c1b802741cca3eb860e859913c7 P 17a50daf9a634b28a4a763ec9099c76a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:10:41.088668 18173 tablet_bootstrap.cc:654] T 5f9d5c1b802741cca3eb860e859913c7 P c1dc1c7cadc94204a8345df0f9140d83: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:41.088977 18174 tablet_bootstrap.cc:492] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Bootstrap starting.
I20260929 19:10:41.089324 18173 tablet_bootstrap.cc:492] T 5f9d5c1b802741cca3eb860e859913c7 P c1dc1c7cadc94204a8345df0f9140d83: No bootstrap required, opened a new log
I20260929 19:10:41.089332 18174 tablet_bootstrap.cc:654] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:41.089382 18173 ts_tablet_manager.cc:1397] T 5f9d5c1b802741cca3eb860e859913c7 P c1dc1c7cadc94204a8345df0f9140d83: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:41.089524 18173 raft_consensus.cc:348] T 5f9d5c1b802741cca3eb860e859913c7 P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.089609 18173 raft_consensus.cc:374] T 5f9d5c1b802741cca3eb860e859913c7 P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:41.089633 18173 raft_consensus.cc:729] T 5f9d5c1b802741cca3eb860e859913c7 P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1dc1c7cadc94204a8345df0f9140d83, State: Initialized, Role: FOLLOWER
I20260929 19:10:41.089687 18173 consensus_queue.cc:260] T 5f9d5c1b802741cca3eb860e859913c7 P c1dc1c7cadc94204a8345df0f9140d83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.089793 18173 ts_tablet_manager.cc:1428] T 5f9d5c1b802741cca3eb860e859913c7 P c1dc1c7cadc94204a8345df0f9140d83: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 19:10:41.089821 18178 tablet_replica.cc:439] T 5f9d5c1b802741cca3eb860e859913c7 P c1dc1c7cadc94204a8345df0f9140d83: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } } }
I20260929 19:10:41.089895 18178 tablet_replica.cc:442] T 5f9d5c1b802741cca3eb860e859913c7 P c1dc1c7cadc94204a8345df0f9140d83: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:10:41.090165 18174 tablet_bootstrap.cc:492] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f: No bootstrap required, opened a new log
I20260929 19:10:41.090217 18174 ts_tablet_manager.cc:1397] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:41.090322 18174 raft_consensus.cc:348] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.090391 18174 raft_consensus.cc:374] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:41.090408 18174 raft_consensus.cc:729] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ebaccd8c40746b28cdbbc2902fd6d0f, State: Initialized, Role: FOLLOWER
I20260929 19:10:41.090456 18174 consensus_queue.cc:260] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.090548 18174 ts_tablet_manager.cc:1428] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:41.090559 18181 tablet_replica.cc:439] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } } }
I20260929 19:10:41.090633 18181 tablet_replica.cc:442] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:10:41.273299 18181 raft_consensus.cc:482] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:10:41.273396 18181 raft_consensus.cc:504] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.273517 18181 leader_election.cc:288] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1dc1c7cadc94204a8345df0f9140d83 (127.9.236.1:36447), 17a50daf9a634b28a4a763ec9099c76a (127.9.236.3:34375)
I20260929 19:10:41.273769 17817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f9d5c1b802741cca3eb860e859913c7" candidate_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1dc1c7cadc94204a8345df0f9140d83" is_pre_election: true
I20260929 19:10:41.273797 18097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f9d5c1b802741cca3eb860e859913c7" candidate_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17a50daf9a634b28a4a763ec9099c76a" is_pre_election: true
I20260929 19:10:41.273878 17817 raft_consensus.cc:2454] T 5f9d5c1b802741cca3eb860e859913c7 P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ebaccd8c40746b28cdbbc2902fd6d0f in term 0.
I20260929 19:10:41.273883 18097 raft_consensus.cc:2454] T 5f9d5c1b802741cca3eb860e859913c7 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ebaccd8c40746b28cdbbc2902fd6d0f in term 0.
I20260929 19:10:41.274052 17888 leader_election.cc:302] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17a50daf9a634b28a4a763ec9099c76a, 6ebaccd8c40746b28cdbbc2902fd6d0f; no voters:
I20260929 19:10:41.274158 18181 raft_consensus.cc:2781] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:10:41.274188 18181 raft_consensus.cc:482] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:10:41.274204 18181 raft_consensus.cc:3037] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:41.274823 18181 raft_consensus.cc:504] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.274920 18181 leader_election.cc:288] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f [CANDIDATE]: Term 1 election: Requested vote from peers c1dc1c7cadc94204a8345df0f9140d83 (127.9.236.1:36447), 17a50daf9a634b28a4a763ec9099c76a (127.9.236.3:34375)
I20260929 19:10:41.275107 17817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f9d5c1b802741cca3eb860e859913c7" candidate_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1dc1c7cadc94204a8345df0f9140d83"
I20260929 19:10:41.275177 17817 raft_consensus.cc:3037] T 5f9d5c1b802741cca3eb860e859913c7 P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:41.275139 18097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f9d5c1b802741cca3eb860e859913c7" candidate_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17a50daf9a634b28a4a763ec9099c76a"
I20260929 19:10:41.275283 18097 raft_consensus.cc:3037] T 5f9d5c1b802741cca3eb860e859913c7 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:41.275794 17817 raft_consensus.cc:2454] T 5f9d5c1b802741cca3eb860e859913c7 P c1dc1c7cadc94204a8345df0f9140d83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ebaccd8c40746b28cdbbc2902fd6d0f in term 1.
I20260929 19:10:41.275960 17888 leader_election.cc:302] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ebaccd8c40746b28cdbbc2902fd6d0f, c1dc1c7cadc94204a8345df0f9140d83; no voters:
I20260929 19:10:41.276053 18181 raft_consensus.cc:2781] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:41.276081 18097 raft_consensus.cc:2454] T 5f9d5c1b802741cca3eb860e859913c7 P 17a50daf9a634b28a4a763ec9099c76a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ebaccd8c40746b28cdbbc2902fd6d0f in term 1.
I20260929 19:10:41.276106 18181 raft_consensus.cc:686] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 LEADER]: Becoming Leader. State: Replica: 6ebaccd8c40746b28cdbbc2902fd6d0f, State: Running, Role: LEADER
I20260929 19:10:41.276165 18181 consensus_queue.cc:237] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.276409 18179 tablet_replica.cc:439] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f: TxnStatusTablet state changed. Reason: New leader 6ebaccd8c40746b28cdbbc2902fd6d0f. Latest consensus state: current_term: 1 leader_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } } }
I20260929 19:10:41.276486 18179 tablet_replica.cc:442] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f: This TxnStatusTablet replica's current role is: LEADER
I20260929 19:10:41.276612 18211 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 19:10:41.276738 18211 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 19:10:41.276870 17658 catalog_manager.cc:5495] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ebaccd8c40746b28cdbbc2902fd6d0f (127.9.236.2). New cstate: current_term: 1 leader_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } health_report { overall_health: UNKNOWN } } }
I20260929 19:10:41.320704 18214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:41.317897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35992 (local address 127.9.236.2:45039)
0929 19:10:41.318019 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:41.318021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:41.318032 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:10:41.318054 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:41.318054 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:41.318063 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:41.318136 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:41.318276 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.318780 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.319943 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.320166 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.320227 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:41.320439 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:41.320474 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:41.320540 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:41.320606 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 19:10:41.320627 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":61,"threads_started":1}
I20260929 19:10:41.320757 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:41.317837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:45039 (local address 127.0.0.1:35992)
0929 19:10:41.317913 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:41.317926 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:41.317971 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:41.318146 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:41.318147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:41.318149 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:10:41.318220 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:41.318224 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.318856 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.318857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:41.319855 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:41.319858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.320240 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.320241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:41.320307 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:41.320325 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:10:41.320551 (+ 226us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:10:41.320557 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 19:10:41.320605 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 19:10:41.320657 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260929 19:10:41.321784 17817 raft_consensus.cc:1261] T 5f9d5c1b802741cca3eb860e859913c7 P c1dc1c7cadc94204a8345df0f9140d83 [term 1 FOLLOWER]: Refusing update from remote peer 6ebaccd8c40746b28cdbbc2902fd6d0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:41.321811 18097 raft_consensus.cc:1261] T 5f9d5c1b802741cca3eb860e859913c7 P 17a50daf9a634b28a4a763ec9099c76a [term 1 FOLLOWER]: Refusing update from remote peer 6ebaccd8c40746b28cdbbc2902fd6d0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:41.322018 18181 consensus_queue.cc:1035] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:41.322146 18181 consensus_queue.cc:1035] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:41.322625 18175 tablet_replica.cc:439] T 5f9d5c1b802741cca3eb860e859913c7 P 17a50daf9a634b28a4a763ec9099c76a: TxnStatusTablet state changed. Reason: New leader 6ebaccd8c40746b28cdbbc2902fd6d0f. Latest consensus state: current_term: 1 leader_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } } }
I20260929 19:10:41.322674 18178 tablet_replica.cc:439] T 5f9d5c1b802741cca3eb860e859913c7 P c1dc1c7cadc94204a8345df0f9140d83: TxnStatusTablet state changed. Reason: New leader 6ebaccd8c40746b28cdbbc2902fd6d0f. Latest consensus state: current_term: 1 leader_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } } }
I20260929 19:10:41.322741 18178 tablet_replica.cc:442] T 5f9d5c1b802741cca3eb860e859913c7 P c1dc1c7cadc94204a8345df0f9140d83: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:10:41.322741 18175 tablet_replica.cc:442] T 5f9d5c1b802741cca3eb860e859913c7 P 17a50daf9a634b28a4a763ec9099c76a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:10:41.323493 18181 tablet_replica.cc:439] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } } }
I20260929 19:10:41.323580 18181 tablet_replica.cc:442] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f: This TxnStatusTablet replica's current role is: LEADER
I20260929 19:10:41.323621 18181 tablet_replica.cc:439] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } } }
I20260929 19:10:41.323661 18181 tablet_replica.cc:442] T 5f9d5c1b802741cca3eb860e859913c7 P 6ebaccd8c40746b28cdbbc2902fd6d0f: This TxnStatusTablet replica's current role is: LEADER
I20260929 19:10:41.324265 18178 tablet_replica.cc:439] T 5f9d5c1b802741cca3eb860e859913c7 P c1dc1c7cadc94204a8345df0f9140d83: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } } }
I20260929 19:10:41.324347 18178 tablet_replica.cc:442] T 5f9d5c1b802741cca3eb860e859913c7 P c1dc1c7cadc94204a8345df0f9140d83: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:10:41.324414 18175 tablet_replica.cc:439] T 5f9d5c1b802741cca3eb860e859913c7 P 17a50daf9a634b28a4a763ec9099c76a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } } }
I20260929 19:10:41.324484 18175 tablet_replica.cc:442] T 5f9d5c1b802741cca3eb860e859913c7 P 17a50daf9a634b28a4a763ec9099c76a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 19:10:41.324796 18217 mvcc.cc:205] Tried to move back new op lower bound from 7334744233252376576 to 7334744233067532288. Current Snapshot: MvccSnapshot[applied={T|T < 7334744233252376576}]
I20260929 19:10:41.333032 18214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:41.329593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:34139 (local address 127.9.236.2:45039)
0929 19:10:41.329665 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:41.329667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:41.329921 (+ 254us) server_negotiation.cc:407] Connection header received
0929 19:10:41.329992 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:41.329993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:41.330004 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:41.330082 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:41.330255 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.330953 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.331910 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.332313 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.332448 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:41.332620 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:41.332660 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:41.332729 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:41.332861 (+ 132us) server_negotiation.cc:299] Negotiation successful
0929 19:10:41.332905 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":22}
I20260929 19:10:41.333119 18226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:41.329529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:45039 (local address 127.9.236.2:34139)
0929 19:10:41.329798 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:41.329834 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:41.329927 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:41.330094 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:41.330095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:41.330097 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:10:41.330185 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:41.330187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.331062 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.331064 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:41.331817 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:41.331820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.332424 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.332425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:41.332489 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:41.332509 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:10:41.332768 (+ 259us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:10:41.332769 (+ 1us) client_negotiation.cc:769] Sending connection context
0929 19:10:41.332837 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 19:10:41.332911 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":4,"thread_start_us":95,"threads_started":1}
I20260929 19:10:41.335712 18097 raft_consensus.cc:1261] T 54b3c484dcd4462cba4e4d9b39f85212 P 17a50daf9a634b28a4a763ec9099c76a [term 1 FOLLOWER]: Refusing update from remote peer 6ebaccd8c40746b28cdbbc2902fd6d0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:41.335752 17817 raft_consensus.cc:1261] T 54b3c484dcd4462cba4e4d9b39f85212 P c1dc1c7cadc94204a8345df0f9140d83 [term 1 FOLLOWER]: Refusing update from remote peer 6ebaccd8c40746b28cdbbc2902fd6d0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:41.335922 18220 consensus_queue.cc:1035] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:41.336061 18179 consensus_queue.cc:1035] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:41.348033 17659 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:41.348177 17659 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:41.348927 17659 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd: Sending DeleteTablet for 3 replicas of tablet 54b3c484dcd4462cba4e4d9b39f85212
I20260929 19:10:41.349229 17936 tablet_service.cc:1514] Processing DeleteTablet for tablet 54b3c484dcd4462cba4e4d9b39f85212 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:41 UTC) from {username='oryx', principal='oryx/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:35980
I20260929 19:10:41.349267 18077 tablet_service.cc:1514] Processing DeleteTablet for tablet 54b3c484dcd4462cba4e4d9b39f85212 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:41 UTC) from {username='oryx', principal='oryx/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:40404
I20260929 19:10:41.349584 17791 tablet_service.cc:1514] Processing DeleteTablet for tablet 54b3c484dcd4462cba4e4d9b39f85212 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:41 UTC) from {username='oryx', principal='oryx/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:34902
I20260929 19:10:41.349763 18231 tablet_replica.cc:330] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f: stopping tablet replica
I20260929 19:10:41.349817 18231 raft_consensus.cc:2229] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:10:41.349876 18231 raft_consensus.cc:2258] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:41.349956 18233 tablet_replica.cc:330] T 54b3c484dcd4462cba4e4d9b39f85212 P c1dc1c7cadc94204a8345df0f9140d83: stopping tablet replica
I20260929 19:10:41.349994 18233 raft_consensus.cc:2229] T 54b3c484dcd4462cba4e4d9b39f85212 P c1dc1c7cadc94204a8345df0f9140d83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:41.350021 18233 raft_consensus.cc:2258] T 54b3c484dcd4462cba4e4d9b39f85212 P c1dc1c7cadc94204a8345df0f9140d83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:41.350270 18233 ts_tablet_manager.cc:1905] T 54b3c484dcd4462cba4e4d9b39f85212 P c1dc1c7cadc94204a8345df0f9140d83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:41.352005 18231 ts_tablet_manager.cc:1905] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:41.353431 18231 ts_tablet_manager.cc:1918] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:10:41.353431 18233 ts_tablet_manager.cc:1918] T 54b3c484dcd4462cba4e4d9b39f85212 P c1dc1c7cadc94204a8345df0f9140d83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:10:41.353483 18233 log.cc:1196] T 54b3c484dcd4462cba4e4d9b39f85212 P c1dc1c7cadc94204a8345df0f9140d83: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-0/wal/wals/54b3c484dcd4462cba4e4d9b39f85212
I20260929 19:10:41.353612 18231 log.cc:1196] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/54b3c484dcd4462cba4e4d9b39f85212
I20260929 19:10:41.353698 18233 ts_tablet_manager.cc:1939] T 54b3c484dcd4462cba4e4d9b39f85212 P c1dc1c7cadc94204a8345df0f9140d83: Deleting consensus metadata
I20260929 19:10:41.353886 18231 ts_tablet_manager.cc:1939] T 54b3c484dcd4462cba4e4d9b39f85212 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Deleting consensus metadata
I20260929 19:10:41.353981 18232 tablet_replica.cc:330] T 54b3c484dcd4462cba4e4d9b39f85212 P 17a50daf9a634b28a4a763ec9099c76a: stopping tablet replica
I20260929 19:10:41.354044 18232 raft_consensus.cc:2229] T 54b3c484dcd4462cba4e4d9b39f85212 P 17a50daf9a634b28a4a763ec9099c76a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:41.354084 18232 raft_consensus.cc:2258] T 54b3c484dcd4462cba4e4d9b39f85212 P 17a50daf9a634b28a4a763ec9099c76a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:41.354079 17644 catalog_manager.cc:4841] TS c1dc1c7cadc94204a8345df0f9140d83 (127.9.236.1:36447): tablet 54b3c484dcd4462cba4e4d9b39f85212 (table test-table [id=978e25abe77745dcaacb8ff9455e2889]) successfully deleted
I20260929 19:10:41.354327 18232 ts_tablet_manager.cc:1905] T 54b3c484dcd4462cba4e4d9b39f85212 P 17a50daf9a634b28a4a763ec9099c76a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:41.354424 17645 catalog_manager.cc:4841] TS 6ebaccd8c40746b28cdbbc2902fd6d0f (127.9.236.2:45039): tablet 54b3c484dcd4462cba4e4d9b39f85212 (table test-table [id=978e25abe77745dcaacb8ff9455e2889]) successfully deleted
I20260929 19:10:41.355581 18232 ts_tablet_manager.cc:1918] T 54b3c484dcd4462cba4e4d9b39f85212 P 17a50daf9a634b28a4a763ec9099c76a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:10:41.355628 18232 log.cc:1196] T 54b3c484dcd4462cba4e4d9b39f85212 P 17a50daf9a634b28a4a763ec9099c76a: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/54b3c484dcd4462cba4e4d9b39f85212
I20260929 19:10:41.355792 18232 ts_tablet_manager.cc:1939] T 54b3c484dcd4462cba4e4d9b39f85212 P 17a50daf9a634b28a4a763ec9099c76a: Deleting consensus metadata
I20260929 19:10:41.356107 17644 catalog_manager.cc:4841] TS 17a50daf9a634b28a4a763ec9099c76a (127.9.236.3:34375): tablet 54b3c484dcd4462cba4e4d9b39f85212 (table test-table [id=978e25abe77745dcaacb8ff9455e2889]) successfully deleted
Sep 29 19:10:41 dist-test-slave-wftq krb5kdc[17615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 19:10:41 dist-test-slave-wftq krb5kdc[17615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:41 dist-test-slave-wftq krb5kdc[17615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
W20260929 19:10:41.376797 18206 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:41.371507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47146 (local address 127.9.236.62:35325)
0929 19:10:41.371554 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:41.371556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:41.371714 (+ 158us) server_negotiation.cc:407] Connection header received
0929 19:10:41.371781 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:41.371782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:41.371797 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:41.371842 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:41.372390 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.372903 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.373698 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.373895 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.373947 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:41.376736 (+ 2789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47146: BlockingRecv error: recv got EOF from 127.0.0.1:47146 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Sep 29 19:10:41 dist-test-slave-wftq krb5kdc[17615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM
I20260929 19:10:41.384629 18206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:41.377801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47156 (local address 127.9.236.62:35325)
0929 19:10:41.377836 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:41.377837 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:41.377896 (+ 59us) server_negotiation.cc:407] Connection header received
0929 19:10:41.377958 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:41.377959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:41.377967 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:41.378006 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:41.378409 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.378846 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.379466 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.379644 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.379693 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:41.382139 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:41.382140 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:41.382140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:41.382146 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:41.383477 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:41.383917 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:41.383917 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:41.383918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:41.383967 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:41.384177 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:41.384178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:41.384178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:41.384272 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:41.384346 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:41.384509 (+ 163us) server_negotiation.cc:299] Negotiation successful
0929 19:10:41.384563 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260929 19:10:41.386349 17659 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:10:41.386607 17659 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:10:41.388046 17936 tablet_service.cc:1467] Processing CreateTablet for tablet 3ec7fa1fcbe0401a94d0bbaa0557e075 (DEFAULT_TABLE table=test-table [id=a564f73e16544880a21015c1e54a92cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:41.388082 17791 tablet_service.cc:1467] Processing CreateTablet for tablet 3ec7fa1fcbe0401a94d0bbaa0557e075 (DEFAULT_TABLE table=test-table [id=a564f73e16544880a21015c1e54a92cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:41.388201 17791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ec7fa1fcbe0401a94d0bbaa0557e075. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:41.388202 17936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ec7fa1fcbe0401a94d0bbaa0557e075. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:41.388201 18077 tablet_service.cc:1467] Processing CreateTablet for tablet 3ec7fa1fcbe0401a94d0bbaa0557e075 (DEFAULT_TABLE table=test-table [id=a564f73e16544880a21015c1e54a92cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:41.388353 18077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ec7fa1fcbe0401a94d0bbaa0557e075. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:41.389571 18173 tablet_bootstrap.cc:492] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P c1dc1c7cadc94204a8345df0f9140d83: Bootstrap starting.
I20260929 19:10:41.389772 18174 tablet_bootstrap.cc:492] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Bootstrap starting.
I20260929 19:10:41.389958 18173 tablet_bootstrap.cc:654] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P c1dc1c7cadc94204a8345df0f9140d83: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:41.389997 18171 tablet_bootstrap.cc:492] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 17a50daf9a634b28a4a763ec9099c76a: Bootstrap starting.
I20260929 19:10:41.390146 18174 tablet_bootstrap.cc:654] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:41.390466 18171 tablet_bootstrap.cc:654] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 17a50daf9a634b28a4a763ec9099c76a: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:41.390859 18173 tablet_bootstrap.cc:492] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P c1dc1c7cadc94204a8345df0f9140d83: No bootstrap required, opened a new log
I20260929 19:10:41.390862 18174 tablet_bootstrap.cc:492] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f: No bootstrap required, opened a new log
I20260929 19:10:41.390905 18174 ts_tablet_manager.cc:1397] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:41.390905 18173 ts_tablet_manager.cc:1397] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P c1dc1c7cadc94204a8345df0f9140d83: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:41.391032 18174 raft_consensus.cc:348] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.391048 18173 raft_consensus.cc:348] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.391109 18173 raft_consensus.cc:374] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:41.391109 18174 raft_consensus.cc:374] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:41.391129 18173 raft_consensus.cc:729] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1dc1c7cadc94204a8345df0f9140d83, State: Initialized, Role: FOLLOWER
I20260929 19:10:41.391129 18174 raft_consensus.cc:729] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ebaccd8c40746b28cdbbc2902fd6d0f, State: Initialized, Role: FOLLOWER
I20260929 19:10:41.391168 18174 consensus_queue.cc:260] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.391168 18173 consensus_queue.cc:260] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P c1dc1c7cadc94204a8345df0f9140d83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.391258 18173 ts_tablet_manager.cc:1428] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P c1dc1c7cadc94204a8345df0f9140d83: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:41.391263 18174 ts_tablet_manager.cc:1428] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:41.391670 18171 tablet_bootstrap.cc:492] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 17a50daf9a634b28a4a763ec9099c76a: No bootstrap required, opened a new log
I20260929 19:10:41.391726 18171 ts_tablet_manager.cc:1397] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 17a50daf9a634b28a4a763ec9099c76a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 19:10:41.391887 18171 raft_consensus.cc:348] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.391961 18171 raft_consensus.cc:374] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:41.391980 18171 raft_consensus.cc:729] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17a50daf9a634b28a4a763ec9099c76a, State: Initialized, Role: FOLLOWER
I20260929 19:10:41.392019 18171 consensus_queue.cc:260] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 17a50daf9a634b28a4a763ec9099c76a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.392123 18171 ts_tablet_manager.cc:1428] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 17a50daf9a634b28a4a763ec9099c76a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:41.559064 18179 raft_consensus.cc:482] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:10:41.559212 18179 raft_consensus.cc:504] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.559410 18179 leader_election.cc:288] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1dc1c7cadc94204a8345df0f9140d83 (127.9.236.1:36447), 17a50daf9a634b28a4a763ec9099c76a (127.9.236.3:34375)
I20260929 19:10:41.559655 17817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec7fa1fcbe0401a94d0bbaa0557e075" candidate_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1dc1c7cadc94204a8345df0f9140d83" is_pre_election: true
I20260929 19:10:41.559720 18097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec7fa1fcbe0401a94d0bbaa0557e075" candidate_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17a50daf9a634b28a4a763ec9099c76a" is_pre_election: true
I20260929 19:10:41.559778 17817 raft_consensus.cc:2454] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ebaccd8c40746b28cdbbc2902fd6d0f in term 0.
I20260929 19:10:41.559810 18097 raft_consensus.cc:2454] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ebaccd8c40746b28cdbbc2902fd6d0f in term 0.
I20260929 19:10:41.560027 17888 leader_election.cc:302] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ebaccd8c40746b28cdbbc2902fd6d0f, c1dc1c7cadc94204a8345df0f9140d83; no voters:
I20260929 19:10:41.560151 18179 raft_consensus.cc:2781] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:10:41.560201 18179 raft_consensus.cc:482] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:10:41.560220 18179 raft_consensus.cc:3037] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:41.561039 18179 raft_consensus.cc:504] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.561159 18179 leader_election.cc:288] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f [CANDIDATE]: Term 1 election: Requested vote from peers c1dc1c7cadc94204a8345df0f9140d83 (127.9.236.1:36447), 17a50daf9a634b28a4a763ec9099c76a (127.9.236.3:34375)
I20260929 19:10:41.561367 17817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec7fa1fcbe0401a94d0bbaa0557e075" candidate_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1dc1c7cadc94204a8345df0f9140d83"
I20260929 19:10:41.561365 18097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ec7fa1fcbe0401a94d0bbaa0557e075" candidate_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17a50daf9a634b28a4a763ec9099c76a"
I20260929 19:10:41.561450 17817 raft_consensus.cc:3037] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:41.561450 18097 raft_consensus.cc:3037] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:41.562219 17817 raft_consensus.cc:2454] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P c1dc1c7cadc94204a8345df0f9140d83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ebaccd8c40746b28cdbbc2902fd6d0f in term 1.
I20260929 19:10:41.562279 18097 raft_consensus.cc:2454] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 17a50daf9a634b28a4a763ec9099c76a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ebaccd8c40746b28cdbbc2902fd6d0f in term 1.
I20260929 19:10:41.562413 17888 leader_election.cc:302] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ebaccd8c40746b28cdbbc2902fd6d0f, c1dc1c7cadc94204a8345df0f9140d83; no voters:
I20260929 19:10:41.562523 18179 raft_consensus.cc:2781] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:41.562592 18179 raft_consensus.cc:686] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 LEADER]: Becoming Leader. State: Replica: 6ebaccd8c40746b28cdbbc2902fd6d0f, State: Running, Role: LEADER
I20260929 19:10:41.562649 18179 consensus_queue.cc:237] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.563372 17658 catalog_manager.cc:5495] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ebaccd8c40746b28cdbbc2902fd6d0f (127.9.236.2). New cstate: current_term: 1 leader_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } health_report { overall_health: UNKNOWN } } }
I20260929 19:10:41.574908 18214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:41.572642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35996 (local address 127.9.236.2:45039)
0929 19:10:41.572683 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:41.572685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:41.572728 (+ 43us) server_negotiation.cc:407] Connection header received
0929 19:10:41.572807 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:41.572808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:41.572820 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:41.572866 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:41.572994 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.573519 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.574365 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.574548 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.574609 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:41.574712 (+ 103us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:41.574739 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:41.574775 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:41.574841 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 19:10:41.574858 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 19:10:41.575596 18097 raft_consensus.cc:1261] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 17a50daf9a634b28a4a763ec9099c76a [term 1 FOLLOWER]: Refusing update from remote peer 6ebaccd8c40746b28cdbbc2902fd6d0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:41.575668 17817 raft_consensus.cc:1261] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P c1dc1c7cadc94204a8345df0f9140d83 [term 1 FOLLOWER]: Refusing update from remote peer 6ebaccd8c40746b28cdbbc2902fd6d0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:41.575845 18181 consensus_queue.cc:1035] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:41.575940 18179 consensus_queue.cc:1035] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:41.578778 17659 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:41.578881 17659 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:41.579473 17659 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd: Sending DeleteTablet for 3 replicas of tablet 3ec7fa1fcbe0401a94d0bbaa0557e075
I20260929 19:10:41.579751 18077 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ec7fa1fcbe0401a94d0bbaa0557e075 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:41 UTC) from {username='oryx', principal='oryx/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:40404
I20260929 19:10:41.579753 17791 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ec7fa1fcbe0401a94d0bbaa0557e075 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:41 UTC) from {username='oryx', principal='oryx/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:34902
I20260929 19:10:41.579779 17936 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ec7fa1fcbe0401a94d0bbaa0557e075 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:41 UTC) from {username='oryx', principal='oryx/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:35980
I20260929 19:10:41.579850 18232 tablet_replica.cc:330] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 17a50daf9a634b28a4a763ec9099c76a: stopping tablet replica
I20260929 19:10:41.579850 18231 tablet_replica.cc:330] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f: stopping tablet replica
I20260929 19:10:41.579891 18232 raft_consensus.cc:2229] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 17a50daf9a634b28a4a763ec9099c76a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:41.579901 18231 raft_consensus.cc:2229] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:10:41.579927 18232 raft_consensus.cc:2258] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 17a50daf9a634b28a4a763ec9099c76a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:41.579958 18231 raft_consensus.cc:2258] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:41.580013 18233 tablet_replica.cc:330] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P c1dc1c7cadc94204a8345df0f9140d83: stopping tablet replica
I20260929 19:10:41.580057 18233 raft_consensus.cc:2229] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P c1dc1c7cadc94204a8345df0f9140d83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:41.580093 18233 raft_consensus.cc:2258] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P c1dc1c7cadc94204a8345df0f9140d83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:41.580078 17659 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:10:41.580243 17659 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:10:41.580451 18232 ts_tablet_manager.cc:1905] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 17a50daf9a634b28a4a763ec9099c76a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:41.580703 18233 ts_tablet_manager.cc:1905] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P c1dc1c7cadc94204a8345df0f9140d83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:41.581305 18231 ts_tablet_manager.cc:1905] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:41.582099 18233 ts_tablet_manager.cc:1918] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P c1dc1c7cadc94204a8345df0f9140d83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:10:41.582149 18233 log.cc:1196] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P c1dc1c7cadc94204a8345df0f9140d83: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-0/wal/wals/3ec7fa1fcbe0401a94d0bbaa0557e075
I20260929 19:10:41.582351 18233 ts_tablet_manager.cc:1939] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P c1dc1c7cadc94204a8345df0f9140d83: Deleting consensus metadata
I20260929 19:10:41.582361 18232 ts_tablet_manager.cc:1918] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 17a50daf9a634b28a4a763ec9099c76a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:10:41.582401 18232 log.cc:1196] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 17a50daf9a634b28a4a763ec9099c76a: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/3ec7fa1fcbe0401a94d0bbaa0557e075
I20260929 19:10:41.582645 18232 ts_tablet_manager.cc:1939] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 17a50daf9a634b28a4a763ec9099c76a: Deleting consensus metadata
I20260929 19:10:41.582829 18231 ts_tablet_manager.cc:1918] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:10:41.582882 18231 log.cc:1196] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/3ec7fa1fcbe0401a94d0bbaa0557e075
I20260929 19:10:41.582856 17936 tablet_service.cc:1467] Processing CreateTablet for tablet afd2c5627aa04620b8f7e821c584d6f3 (DEFAULT_TABLE table=test-table [id=87c9ddaa5c504aea82c4ed08ba7cce76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:41.582985 17936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afd2c5627aa04620b8f7e821c584d6f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:41.582981 18077 tablet_service.cc:1467] Processing CreateTablet for tablet afd2c5627aa04620b8f7e821c584d6f3 (DEFAULT_TABLE table=test-table [id=87c9ddaa5c504aea82c4ed08ba7cce76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:41.583057 18231 ts_tablet_manager.cc:1939] T 3ec7fa1fcbe0401a94d0bbaa0557e075 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Deleting consensus metadata
I20260929 19:10:41.583060 18077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afd2c5627aa04620b8f7e821c584d6f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:41.583312 17791 tablet_service.cc:1467] Processing CreateTablet for tablet afd2c5627aa04620b8f7e821c584d6f3 (DEFAULT_TABLE table=test-table [id=87c9ddaa5c504aea82c4ed08ba7cce76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:10:41.583349 17645 catalog_manager.cc:4841] TS 6ebaccd8c40746b28cdbbc2902fd6d0f (127.9.236.2:45039): tablet 3ec7fa1fcbe0401a94d0bbaa0557e075 (table test-table [id=a564f73e16544880a21015c1e54a92cf]) successfully deleted
I20260929 19:10:41.583423 17791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afd2c5627aa04620b8f7e821c584d6f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:41.584117 18171 tablet_bootstrap.cc:492] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a: Bootstrap starting.
I20260929 19:10:41.584300 18173 tablet_bootstrap.cc:492] T afd2c5627aa04620b8f7e821c584d6f3 P c1dc1c7cadc94204a8345df0f9140d83: Bootstrap starting.
I20260929 19:10:41.584614 18174 tablet_bootstrap.cc:492] T afd2c5627aa04620b8f7e821c584d6f3 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Bootstrap starting.
I20260929 19:10:41.584758 18171 tablet_bootstrap.cc:654] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:41.584781 18173 tablet_bootstrap.cc:654] T afd2c5627aa04620b8f7e821c584d6f3 P c1dc1c7cadc94204a8345df0f9140d83: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:41.584949 18174 tablet_bootstrap.cc:654] T afd2c5627aa04620b8f7e821c584d6f3 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:41.585134 17644 catalog_manager.cc:4841] TS 17a50daf9a634b28a4a763ec9099c76a (127.9.236.3:34375): tablet 3ec7fa1fcbe0401a94d0bbaa0557e075 (table test-table [id=a564f73e16544880a21015c1e54a92cf]) successfully deleted
I20260929 19:10:41.585635 18174 tablet_bootstrap.cc:492] T afd2c5627aa04620b8f7e821c584d6f3 P 6ebaccd8c40746b28cdbbc2902fd6d0f: No bootstrap required, opened a new log
I20260929 19:10:41.585683 18174 ts_tablet_manager.cc:1397] T afd2c5627aa04620b8f7e821c584d6f3 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 19:10:41.585807 18174 raft_consensus.cc:348] T afd2c5627aa04620b8f7e821c584d6f3 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.585841 17644 catalog_manager.cc:4841] TS c1dc1c7cadc94204a8345df0f9140d83 (127.9.236.1:36447): tablet 3ec7fa1fcbe0401a94d0bbaa0557e075 (table test-table [id=a564f73e16544880a21015c1e54a92cf]) successfully deleted
I20260929 19:10:41.585879 18174 raft_consensus.cc:374] T afd2c5627aa04620b8f7e821c584d6f3 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:41.585902 18174 raft_consensus.cc:729] T afd2c5627aa04620b8f7e821c584d6f3 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ebaccd8c40746b28cdbbc2902fd6d0f, State: Initialized, Role: FOLLOWER
I20260929 19:10:41.585947 18174 consensus_queue.cc:260] T afd2c5627aa04620b8f7e821c584d6f3 P 6ebaccd8c40746b28cdbbc2902fd6d0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.586024 18173 tablet_bootstrap.cc:492] T afd2c5627aa04620b8f7e821c584d6f3 P c1dc1c7cadc94204a8345df0f9140d83: No bootstrap required, opened a new log
I20260929 19:10:41.586073 18174 ts_tablet_manager.cc:1428] T afd2c5627aa04620b8f7e821c584d6f3 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:41.586090 18173 ts_tablet_manager.cc:1397] T afd2c5627aa04620b8f7e821c584d6f3 P c1dc1c7cadc94204a8345df0f9140d83: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:10:41.586264 18171 tablet_bootstrap.cc:492] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a: No bootstrap required, opened a new log
I20260929 19:10:41.586232 18173 raft_consensus.cc:348] T afd2c5627aa04620b8f7e821c584d6f3 P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.586301 18173 raft_consensus.cc:374] T afd2c5627aa04620b8f7e821c584d6f3 P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:41.586311 18171 ts_tablet_manager.cc:1397] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 19:10:41.586321 18173 raft_consensus.cc:729] T afd2c5627aa04620b8f7e821c584d6f3 P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1dc1c7cadc94204a8345df0f9140d83, State: Initialized, Role: FOLLOWER
I20260929 19:10:41.586366 18173 consensus_queue.cc:260] T afd2c5627aa04620b8f7e821c584d6f3 P c1dc1c7cadc94204a8345df0f9140d83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.586462 18173 ts_tablet_manager.cc:1428] T afd2c5627aa04620b8f7e821c584d6f3 P c1dc1c7cadc94204a8345df0f9140d83: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260929 19:10:41.586444 18171 raft_consensus.cc:348] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.586498 18171 raft_consensus.cc:374] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:41.586515 18171 raft_consensus.cc:729] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17a50daf9a634b28a4a763ec9099c76a, State: Initialized, Role: FOLLOWER
I20260929 19:10:41.586552 18171 consensus_queue.cc:260] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.586622 18171 ts_tablet_manager.cc:1428] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:41.639475 18175 raft_consensus.cc:482] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:10:41.639576 18175 raft_consensus.cc:504] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.639861 18175 leader_election.cc:288] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1dc1c7cadc94204a8345df0f9140d83 (127.9.236.1:36447), 6ebaccd8c40746b28cdbbc2902fd6d0f (127.9.236.2:45039)
I20260929 19:10:41.643160 18258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:41.640225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:39749 (local address 127.9.236.1:36447)
0929 19:10:41.640372 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:41.640374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:41.640384 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:10:41.640413 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:41.640413 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:41.640422 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:41.640496 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:41.640696 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.641304 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.642407 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.642797 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.642861 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:41.642875 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:41.643036 (+ 161us) server_negotiation.cc:299] Negotiation successful
0929 19:10:41.643081 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":62,"threads_started":1}
I20260929 19:10:41.643151 18214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:41.640180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:39493 (local address 127.9.236.2:45039)
0929 19:10:41.640229 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:41.640232 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:41.640283 (+ 51us) server_negotiation.cc:407] Connection header received
0929 19:10:41.640387 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:41.640389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:41.640400 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:41.640457 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:10:41.640658 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.641241 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.642400 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.642787 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.642848 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:41.642862 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:41.643018 (+ 156us) server_negotiation.cc:299] Negotiation successful
0929 19:10:41.643063 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260929 19:10:41.643186 18256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:41.640091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:36447 (local address 127.9.236.3:39749)
0929 19:10:41.640278 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:41.640290 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:41.640359 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:41.640513 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:41.640514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:41.640517 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:41.640613 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:41.640615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.641391 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.641392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:41.642314 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:41.642316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.642903 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.642904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:41.642965 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:41.642969 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:10:41.643020 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 19:10:41.643070 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":2,"thread_start_us":79,"threads_started":1}
I20260929 19:10:41.643209 18257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:41.640058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:45039 (local address 127.9.236.3:39493)
0929 19:10:41.640224 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:41.640236 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:41.640302 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:41.640481 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:41.640482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:41.640486 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:10:41.640586 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:41.640588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.641348 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.641349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:41.642311 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:41.642313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.642869 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.642869 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:41.642947 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:41.642951 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:10:41.642997 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 19:10:41.643048 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":55,"thread_start_us":51,"threads_started":1}
I20260929 19:10:41.643467 17957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd2c5627aa04620b8f7e821c584d6f3" candidate_uuid: "17a50daf9a634b28a4a763ec9099c76a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" is_pre_election: true
I20260929 19:10:41.643496 17817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd2c5627aa04620b8f7e821c584d6f3" candidate_uuid: "17a50daf9a634b28a4a763ec9099c76a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1dc1c7cadc94204a8345df0f9140d83" is_pre_election: true
I20260929 19:10:41.643620 17817 raft_consensus.cc:2454] T afd2c5627aa04620b8f7e821c584d6f3 P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17a50daf9a634b28a4a763ec9099c76a in term 0.
I20260929 19:10:41.643625 17957 raft_consensus.cc:2454] T afd2c5627aa04620b8f7e821c584d6f3 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17a50daf9a634b28a4a763ec9099c76a in term 0.
I20260929 19:10:41.643846 18028 leader_election.cc:302] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17a50daf9a634b28a4a763ec9099c76a, c1dc1c7cadc94204a8345df0f9140d83; no voters:
I20260929 19:10:41.643987 18175 raft_consensus.cc:2781] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:10:41.644032 18175 raft_consensus.cc:482] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:10:41.644078 18175 raft_consensus.cc:3037] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:41.644824 18175 raft_consensus.cc:504] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.644965 18175 leader_election.cc:288] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a [CANDIDATE]: Term 1 election: Requested vote from peers c1dc1c7cadc94204a8345df0f9140d83 (127.9.236.1:36447), 6ebaccd8c40746b28cdbbc2902fd6d0f (127.9.236.2:45039)
I20260929 19:10:41.645083 17817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd2c5627aa04620b8f7e821c584d6f3" candidate_uuid: "17a50daf9a634b28a4a763ec9099c76a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1dc1c7cadc94204a8345df0f9140d83"
I20260929 19:10:41.645221 17817 raft_consensus.cc:3037] T afd2c5627aa04620b8f7e821c584d6f3 P c1dc1c7cadc94204a8345df0f9140d83 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:41.645187 17957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd2c5627aa04620b8f7e821c584d6f3" candidate_uuid: "17a50daf9a634b28a4a763ec9099c76a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f"
I20260929 19:10:41.645257 17957 raft_consensus.cc:3037] T afd2c5627aa04620b8f7e821c584d6f3 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:41.645913 17817 raft_consensus.cc:2454] T afd2c5627aa04620b8f7e821c584d6f3 P c1dc1c7cadc94204a8345df0f9140d83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17a50daf9a634b28a4a763ec9099c76a in term 1.
I20260929 19:10:41.645979 17957 raft_consensus.cc:2454] T afd2c5627aa04620b8f7e821c584d6f3 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17a50daf9a634b28a4a763ec9099c76a in term 1.
I20260929 19:10:41.646097 18028 leader_election.cc:302] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17a50daf9a634b28a4a763ec9099c76a, c1dc1c7cadc94204a8345df0f9140d83; no voters:
I20260929 19:10:41.646201 18175 raft_consensus.cc:2781] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:41.646366 18175 raft_consensus.cc:686] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a [term 1 LEADER]: Becoming Leader. State: Replica: 17a50daf9a634b28a4a763ec9099c76a, State: Running, Role: LEADER
I20260929 19:10:41.646456 18175 consensus_queue.cc:237] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } }
I20260929 19:10:41.647246 17659 catalog_manager.cc:5495] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a reported cstate change: term changed from 0 to 1, leader changed from <none> to 17a50daf9a634b28a4a763ec9099c76a (127.9.236.3). New cstate: current_term: 1 leader_uuid: "17a50daf9a634b28a4a763ec9099c76a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17a50daf9a634b28a4a763ec9099c76a" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34375 } health_report { overall_health: HEALTHY } } }
I20260929 19:10:41.666654 18261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:41.663738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40414 (local address 127.9.236.3:34375)
0929 19:10:41.663933 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:41.663935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:41.663941 (+ 6us) server_negotiation.cc:407] Connection header received
0929 19:10:41.663976 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:41.663977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:41.663999 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:41.664090 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:41.664212 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.664847 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.665833 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.666064 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.666119 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:41.666266 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:41.666403 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:41.666473 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:41.666524 (+ 51us) server_negotiation.cc:299] Negotiation successful
0929 19:10:41.666548 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":120,"threads_started":1}
I20260929 19:10:41.671821 18264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:41.669373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:45039 (local address 127.9.236.3:32849)
0929 19:10:41.669554 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:41.669567 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:41.669634 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:41.669754 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:41.669754 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:41.669756 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:10:41.669829 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:41.669830 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.670437 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.670438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:41.671103 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:41.671105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.671420 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.671420 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:41.671481 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:41.671494 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:10:41.671644 (+ 150us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:10:41.671646 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:10:41.671695 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 19:10:41.671734 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":92,"threads_started":1}
I20260929 19:10:41.671842 18214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:41.669489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:32849 (local address 127.9.236.2:45039)
0929 19:10:41.669566 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:41.669569 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:41.669620 (+ 51us) server_negotiation.cc:407] Connection header received
0929 19:10:41.669685 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:41.669686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:41.669702 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:41.669747 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:41.669883 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.670362 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.671173 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.671359 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.671411 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:41.671567 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:41.671594 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:41.671633 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:41.671731 (+ 98us) server_negotiation.cc:299] Negotiation successful
0929 19:10:41.671773 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 19:10:41.674397 17817 raft_consensus.cc:1261] T afd2c5627aa04620b8f7e821c584d6f3 P c1dc1c7cadc94204a8345df0f9140d83 [term 1 FOLLOWER]: Refusing update from remote peer 17a50daf9a634b28a4a763ec9099c76a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:41.674456 17957 raft_consensus.cc:1261] T afd2c5627aa04620b8f7e821c584d6f3 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 FOLLOWER]: Refusing update from remote peer 17a50daf9a634b28a4a763ec9099c76a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:10:41.674746 18259 consensus_queue.cc:1035] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1dc1c7cadc94204a8345df0f9140d83" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 36447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:41.674908 18175 consensus_queue.cc:1035] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ebaccd8c40746b28cdbbc2902fd6d0f" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:10:41.682574 18226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:41.679826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:34375 (local address 127.9.236.2:51443)
0929 19:10:41.679877 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:41.679890 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:41.679933 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:41.680103 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:41.680104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:41.680106 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 19:10:41.680164 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:41.680165 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.680840 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.680841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:41.681648 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:41.681651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.682114 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.682115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:41.682162 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:41.682171 (+ 9us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 19:10:41.682364 (+ 193us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 19:10:41.682366 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:10:41.682414 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 19:10:41.682478 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260929 19:10:41.682808 18261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:41.679928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:51443 (local address 127.9.236.3:34375)
0929 19:10:41.679973 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:41.679975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:41.679982 (+ 7us) server_negotiation.cc:407] Connection header received
0929 19:10:41.680000 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:41.680001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:41.680010 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:41.680074 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:10:41.680229 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.680733 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.681761 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:41.682019 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:41.682108 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:41.682248 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:10:41.682289 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:10:41.682342 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:41.682703 (+ 361us) server_negotiation.cc:299] Negotiation successful
0929 19:10:41.682740 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260929 19:10:41.693048 17659 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:41.693203 17659 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:10:41.693984 17659 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19f9285149614f4eb38115d95d52e7fd: Sending DeleteTablet for 3 replicas of tablet afd2c5627aa04620b8f7e821c584d6f3
I20260929 19:10:41.694298 17791 tablet_service.cc:1514] Processing DeleteTablet for tablet afd2c5627aa04620b8f7e821c584d6f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:41 UTC) from {username='oryx', principal='oryx/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:34902
I20260929 19:10:41.694315 17936 tablet_service.cc:1514] Processing DeleteTablet for tablet afd2c5627aa04620b8f7e821c584d6f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:41 UTC) from {username='oryx', principal='oryx/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:35980
I20260929 19:10:41.694415 18231 tablet_replica.cc:330] T afd2c5627aa04620b8f7e821c584d6f3 P 6ebaccd8c40746b28cdbbc2902fd6d0f: stopping tablet replica
I20260929 19:10:41.694435 18233 tablet_replica.cc:330] T afd2c5627aa04620b8f7e821c584d6f3 P c1dc1c7cadc94204a8345df0f9140d83: stopping tablet replica
I20260929 19:10:41.694463 18231 raft_consensus.cc:2229] T afd2c5627aa04620b8f7e821c584d6f3 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:41.694504 18231 raft_consensus.cc:2258] T afd2c5627aa04620b8f7e821c584d6f3 P 6ebaccd8c40746b28cdbbc2902fd6d0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:41.694499 18233 raft_consensus.cc:2229] T afd2c5627aa04620b8f7e821c584d6f3 P c1dc1c7cadc94204a8345df0f9140d83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:10:41.694540 18233 raft_consensus.cc:2258] T afd2c5627aa04620b8f7e821c584d6f3 P c1dc1c7cadc94204a8345df0f9140d83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:41.694610 18077 tablet_service.cc:1514] Processing DeleteTablet for tablet afd2c5627aa04620b8f7e821c584d6f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:10:41 UTC) from {username='oryx', principal='oryx/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:40404
I20260929 19:10:41.694715 18232 tablet_replica.cc:330] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a: stopping tablet replica
I20260929 19:10:41.694793 18232 raft_consensus.cc:2229] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:10:41.694903 18232 raft_consensus.cc:2258] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:10:41.695057 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 17732
I20260929 19:10:41.695098 18233 ts_tablet_manager.cc:1905] T afd2c5627aa04620b8f7e821c584d6f3 P c1dc1c7cadc94204a8345df0f9140d83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:41.695231 18231 ts_tablet_manager.cc:1905] T afd2c5627aa04620b8f7e821c584d6f3 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:41.696951 18231 ts_tablet_manager.cc:1918] T afd2c5627aa04620b8f7e821c584d6f3 P 6ebaccd8c40746b28cdbbc2902fd6d0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:10:41.697011 18231 log.cc:1196] T afd2c5627aa04620b8f7e821c584d6f3 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/afd2c5627aa04620b8f7e821c584d6f3
I20260929 19:10:41.697057 18232 ts_tablet_manager.cc:1905] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:10:41.697265 18231 ts_tablet_manager.cc:1939] T afd2c5627aa04620b8f7e821c584d6f3 P 6ebaccd8c40746b28cdbbc2902fd6d0f: Deleting consensus metadata
I20260929 19:10:41.697628 17645 catalog_manager.cc:4841] TS 6ebaccd8c40746b28cdbbc2902fd6d0f (127.9.236.2:45039): tablet afd2c5627aa04620b8f7e821c584d6f3 (table test-table [id=87c9ddaa5c504aea82c4ed08ba7cce76]) successfully deleted
I20260929 19:10:41.698707 18232 ts_tablet_manager.cc:1918] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 19:10:41.698778 18232 log.cc:1196] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/afd2c5627aa04620b8f7e821c584d6f3
I20260929 19:10:41.698985 18232 ts_tablet_manager.cc:1939] T afd2c5627aa04620b8f7e821c584d6f3 P 17a50daf9a634b28a4a763ec9099c76a: Deleting consensus metadata
I20260929 19:10:41.699393 17644 catalog_manager.cc:4841] TS 17a50daf9a634b28a4a763ec9099c76a (127.9.236.3:34375): tablet afd2c5627aa04620b8f7e821c584d6f3 (table test-table [id=87c9ddaa5c504aea82c4ed08ba7cce76]) successfully deleted
W20260929 19:10:41.702543 17644 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.236.1:36447 (error 108)
I20260929 19:10:41.702766 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 17872
I20260929 19:10:41.705204 18271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:41.704423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:36447 (local address 127.0.0.1:34918)
0929 19:10:41.705070 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:41.705153 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.1:36447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":69,"threads_started":1}
W20260929 19:10:41.705364 17644 catalog_manager.cc:4584] TS c1dc1c7cadc94204a8345df0f9140d83 (127.9.236.1:36447): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet afd2c5627aa04620b8f7e821c584d6f3: Network error: Client connection negotiation failed: client connection to 127.9.236.1:36447: connect: Connection refused (error 111)
I20260929 19:10:41.710638 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 18012
I20260929 19:10:41.715503 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 17628
2026-09-29T19:10:41Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2936 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:10:41 dist-test-slave-wftq krb5kdc[18274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:10:41 dist-test-slave-wftq krb5kdc[18274](info): set up 2 sockets
Sep 29 19:10:41 dist-test-slave-wftq krb5kdc[18274](info): commencing operation
krb5kdc: starting...
W20260929 19:10:42.685875 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.949s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:10:42 dist-test-slave-wftq krb5kdc[18274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:10:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:10:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:10:42.807708 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:43871
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:39411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:43871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:42.885550 18287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:42.885764 18287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:42.885794 18287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:42.886920 18287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:42.886960 18287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:42.886979 18287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:42.886996 18287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:42.887010 18287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:42.888437 18287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:39411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:43871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:43871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:42.888671 18287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:42.888870 18287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:42.893567 18295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:42.893692 18293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:42.893810 18287 server_base.cc:1034] running on GCE node
W20260929 19:10:42.893567 18292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:42.894155 18287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:42.894411 18287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:42.895547 18287 hybrid_clock.cc:648] HybridClock initialized: now 1790709042895531 us; error 36 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-29T19:10:42Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1179 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:10:42 dist-test-slave-wftq krb5kdc[18300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:10:42 dist-test-slave-wftq krb5kdc[18300](info): set up 2 sockets
Sep 29 19:10:42 dist-test-slave-wftq krb5kdc[18300](info): commencing operation
krb5kdc: starting...
W20260929 19:10:43.845638 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:10:43 dist-test-slave-wftq krb5kdc[18300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:10:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:10:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:10:43.965241 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:39695
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:46543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:39695
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:44.050316 18313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:44.050546 18313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:44.050567 18313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:44.051678 18313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:44.051714 18313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:44.051726 18313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:44.051738 18313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:44.051748 18313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:44.053202 18313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:46543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:39695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:39695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:44.053409 18313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:44.053596 18313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:44.058697 18321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:44.058854 18313 server_base.cc:1034] running on GCE node
W20260929 19:10:44.058697 18319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:44.058725 18318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:44.059314 18313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:44.059620 18313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:44.060771 18313 hybrid_clock.cc:648] HybridClock initialized: now 1790709044060760 us; error 26 us; skew 500 ppm
Sep 29 19:10:44 dist-test-slave-wftq krb5kdc[18300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709044, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:44.063441 18313 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:10:44.063916 18313 webserver.cc:460] Webserver started at http://127.9.236.62:34835/ using document root <none> and password file <none>
I20260929 19:10:44.064116 18313 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:44.064158 18313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:44.064256 18313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:44.065404 18313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "87d34d67728f45de838744a2e87de2f1"
format_stamp: "Formatted at 2026-09-29 19:10:44 on dist-test-slave-wftq"
server_key: "41c1cf8c696a231c8d6ee1d6cd4f033e"
server_key_iv: "f85a47bd3cacfd3860c00d6b228b5ba3"
server_key_version: "encryptionkey@0"
I20260929 19:10:44.065706 18313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "87d34d67728f45de838744a2e87de2f1"
format_stamp: "Formatted at 2026-09-29 19:10:44 on dist-test-slave-wftq"
server_key: "41c1cf8c696a231c8d6ee1d6cd4f033e"
server_key_iv: "f85a47bd3cacfd3860c00d6b228b5ba3"
server_key_version: "encryptionkey@0"
I20260929 19:10:44.066982 18313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:44.067734 18328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:44.067943 18313 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:44.068032 18313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "87d34d67728f45de838744a2e87de2f1"
format_stamp: "Formatted at 2026-09-29 19:10:44 on dist-test-slave-wftq"
server_key: "41c1cf8c696a231c8d6ee1d6cd4f033e"
server_key_iv: "f85a47bd3cacfd3860c00d6b228b5ba3"
server_key_version: "encryptionkey@0"
I20260929 19:10:44.068099 18313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:44.083377 18313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:44.085455 18313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:44.085598 18313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:44.089716 18313 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:39695
I20260929 19:10:44.089787 18380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:39695 every 8 connection(s)
I20260929 19:10:44.090111 18313 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:10:44.090260 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 18313
I20260929 19:10:44.090417 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:10:44.090665 10160 external_mini_cluster.cc:1428] Setting key 6bebe5a643400936a744cbfce7652914
I20260929 19:10:44.091256 18381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:10:44.094554 18381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1: Bootstrap starting.
I20260929 19:10:44.095283 18381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1: Neither blocks nor log segments found. Creating new log.
I20260929 19:10:44.095597 18381 log.cc:824] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:44.096421 18381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1: No bootstrap required, opened a new log
Sep 29 19:10:44 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:44.098938 18381 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87d34d67728f45de838744a2e87de2f1" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39695 } }
I20260929 19:10:44.099124 18381 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:10:44.099152 18381 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87d34d67728f45de838744a2e87de2f1, State: Initialized, Role: FOLLOWER
I20260929 19:10:44.099270 18381 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87d34d67728f45de838744a2e87de2f1" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39695 } }
I20260929 19:10:44.099349 18381 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:10:44.099375 18381 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:10:44.099412 18381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:10:44.100272 18381 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87d34d67728f45de838744a2e87de2f1" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39695 } }
I20260929 19:10:44.100382 18381 leader_election.cc:302] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87d34d67728f45de838744a2e87de2f1; no voters:
I20260929 19:10:44.100572 18381 leader_election.cc:288] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:10:44.100646 18386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:10:44.100864 18386 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [term 1 LEADER]: Becoming Leader. State: Replica: 87d34d67728f45de838744a2e87de2f1, State: Running, Role: LEADER
I20260929 19:10:44.100898 18381 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:44.101073 18386 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87d34d67728f45de838744a2e87de2f1" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39695 } }
I20260929 19:10:44.102034 18384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:44.091260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55480 (local address 127.9.236.62:39695)
0929 19:10:44.091432 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:44.091441 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:44.091468 (+ 27us) server_negotiation.cc:407] Connection header received
0929 19:10:44.091926 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:44.091933 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:44.091980 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:44.092113 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:44.092903 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:44.093911 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:44.095008 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:44.095469 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:44.095556 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:44.098198 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:44.098203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:44.098204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:44.098218 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:44.100242 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:44.100635 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:44.100638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:44.100639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:44.100806 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:44.101094 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:44.101095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:44.101095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:44.101402 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:44.101532 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:44.101682 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 19:10:44.101821 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":69,"threads_started":1}
I20260929 19:10:44.102066 18387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87d34d67728f45de838744a2e87de2f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87d34d67728f45de838744a2e87de2f1" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39695 } } }
I20260929 19:10:44.102355 18387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:44.102509 18388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87d34d67728f45de838744a2e87de2f1. Latest consensus state: current_term: 1 leader_uuid: "87d34d67728f45de838744a2e87de2f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87d34d67728f45de838744a2e87de2f1" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39695 } } }
I20260929 19:10:44.103243 18388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:44.102636 18402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:44.103794 18402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:10:44.105252 18402 catalog_manager.cc:1347] Generated new cluster ID: 8919099b060c423581f9adeb65c89705
I20260929 19:10:44.105299 18402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:10:44.123190 18402 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:10:44.123724 18402 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:44.134667 18402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1: Generated new TSK 0
I20260929 19:10:44.134846 18402 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.236.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
I20260929 19:10:44.171906 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:39695
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:46543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:44.256939 18407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:44.257174 18407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:44.257196 18407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:44.258355 18407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:44.258400 18407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:44.258425 18407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:10:44.260031 18407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:46543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:39695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:44.260262 18407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:44.260485 18407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:44.265587 18412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:44.265581 18413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:44.265545 18415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:44.265698 18407 server_base.cc:1034] running on GCE node
I20260929 19:10:44.266039 18407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:44.266276 18407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:44.267457 18407 hybrid_clock.cc:648] HybridClock initialized: now 1790709044267435 us; error 35 us; skew 500 ppm
Sep 29 19:10:44 dist-test-slave-wftq krb5kdc[18300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709044, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:44.270025 18407 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:10:44.270514 18407 webserver.cc:460] Webserver started at http://127.9.236.1:32853/ using document root <none> and password file <none>
I20260929 19:10:44.270778 18407 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:44.270828 18407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:44.270951 18407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:44.272006 18407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "716b8d8217f64c3198ace5654fa9550b"
format_stamp: "Formatted at 2026-09-29 19:10:44 on dist-test-slave-wftq"
server_key: "f7cdabf5d746ca66e81b5372d93f0475"
server_key_iv: "220d3958566a3c3fd23b96168ec6cc40"
server_key_version: "encryptionkey@0"
I20260929 19:10:44.272413 18407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "716b8d8217f64c3198ace5654fa9550b"
format_stamp: "Formatted at 2026-09-29 19:10:44 on dist-test-slave-wftq"
server_key: "f7cdabf5d746ca66e81b5372d93f0475"
server_key_iv: "220d3958566a3c3fd23b96168ec6cc40"
server_key_version: "encryptionkey@0"
I20260929 19:10:44.273792 18407 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:10:44.274681 18422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:44.274884 18407 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:44.274969 18407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "716b8d8217f64c3198ace5654fa9550b"
format_stamp: "Formatted at 2026-09-29 19:10:44 on dist-test-slave-wftq"
server_key: "f7cdabf5d746ca66e81b5372d93f0475"
server_key_iv: "220d3958566a3c3fd23b96168ec6cc40"
server_key_version: "encryptionkey@0"
I20260929 19:10:44.275061 18407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:44.287599 18407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:44.289489 18407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:44.289618 18407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:44.289825 18407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:44.290110 18407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:44.290143 18407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:44.290171 18407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:44.290187 18407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:44.296291 18407 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:38825
I20260929 19:10:44.296352 18535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:38825 every 8 connection(s)
I20260929 19:10:44.296622 18407 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:10:44.297628 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 18407
I20260929 19:10:44.297742 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:10:44.297933 10160 external_mini_cluster.cc:1428] Setting key dde781dffd6ce04cc2317958f3152e5f
Sep 29 19:10:44 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709044, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.236.2@KRBTEST.COM; defaulting to no policy
I20260929 19:10:44.306591 18384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:44.297446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:57823 (local address 127.9.236.62:39695)
0929 19:10:44.297509 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:44.297511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:44.297643 (+ 132us) server_negotiation.cc:407] Connection header received
0929 19:10:44.298210 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:44.298211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:44.298224 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:44.298311 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:44.299338 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:44.300090 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:44.301572 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:44.301863 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:44.301946 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:44.304015 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:44.304016 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:44.304017 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:44.304024 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:44.305375 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:44.305740 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:44.305741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:44.305741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:44.305792 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:44.305999 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:44.306000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:44.306000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:44.306111 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:44.306166 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:44.306406 (+ 240us) server_negotiation.cc:299] Negotiation successful
0929 19:10:44.306490 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 19:10:44.306728 18538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:44.297340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.1:57823)
0929 19:10:44.297509 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:44.297550 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:44.298089 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:44.298412 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:44.298416 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:44.299033 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:44.299223 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:44.299228 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:44.300213 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:44.300215 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:44.301480 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:44.301483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:44.301935 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:44.301936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:44.302073 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:44.302533 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:44.302542 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:44.303932 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:44.305448 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:44.305454 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:44.305456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:44.305677 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:44.305845 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:44.305846 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:44.305847 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:44.305939 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:44.306164 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:44.306170 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:44.306296 (+ 126us) client_negotiation.cc:769] Sending connection context
0929 19:10:44.306380 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 19:10:44.306522 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260929 19:10:44.307338 18536 heartbeater.cc:347] Connected to a master server at 127.9.236.62:39695
I20260929 19:10:44.307461 18536 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:44.307713 18536 heartbeater.cc:511] Master 127.9.236.62:39695 requested a full tablet report, sending...
I20260929 19:10:44.308300 18345 ts_manager.cc:194] Registered new tserver with Master: 716b8d8217f64c3198ace5654fa9550b (127.9.236.1:38825)
I20260929 19:10:44.308921 18345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:57823
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
I20260929 19:10:44.324784 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:39695
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:46543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:44.410434 18541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:44.410648 18541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:44.410670 18541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:44.411798 18541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:44.411832 18541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:44.411856 18541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:10:44.413283 18541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:46543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:39695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:44.413493 18541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:44.413682 18541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:44.418723 18549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:44.418800 18547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:44.418730 18546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:44.419031 18541 server_base.cc:1034] running on GCE node
I20260929 19:10:44.419274 18541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:44.419533 18541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:44.420671 18541 hybrid_clock.cc:648] HybridClock initialized: now 1790709044420661 us; error 29 us; skew 500 ppm
Sep 29 19:10:44 dist-test-slave-wftq krb5kdc[18300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709044, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:44.423159 18541 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:10:44.423593 18541 webserver.cc:460] Webserver started at http://127.9.236.2:44521/ using document root <none> and password file <none>
I20260929 19:10:44.423816 18541 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:44.423866 18541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:44.423986 18541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:44.425032 18541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "7ed7f5b865b74849a2a0f37516a9fffe"
format_stamp: "Formatted at 2026-09-29 19:10:44 on dist-test-slave-wftq"
server_key: "cda4e6abe7abdf7b2ce14058cd2ac1a4"
server_key_iv: "5b3999a94e4879bf3bd60d7a6c2c1884"
server_key_version: "encryptionkey@0"
I20260929 19:10:44.425414 18541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "7ed7f5b865b74849a2a0f37516a9fffe"
format_stamp: "Formatted at 2026-09-29 19:10:44 on dist-test-slave-wftq"
server_key: "cda4e6abe7abdf7b2ce14058cd2ac1a4"
server_key_iv: "5b3999a94e4879bf3bd60d7a6c2c1884"
server_key_version: "encryptionkey@0"
I20260929 19:10:44.426702 18541 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:44.427449 18556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:44.427670 18541 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:44.427755 18541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "7ed7f5b865b74849a2a0f37516a9fffe"
format_stamp: "Formatted at 2026-09-29 19:10:44 on dist-test-slave-wftq"
server_key: "cda4e6abe7abdf7b2ce14058cd2ac1a4"
server_key_iv: "5b3999a94e4879bf3bd60d7a6c2c1884"
server_key_version: "encryptionkey@0"
I20260929 19:10:44.427822 18541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:44.448323 18541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:44.450266 18541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:44.450387 18541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:44.450570 18541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:44.450883 18541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:44.450917 18541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:44.450945 18541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:44.450963 18541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:44.457062 18541 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:41947
I20260929 19:10:44.457191 18669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:41947 every 8 connection(s)
I20260929 19:10:44.457435 18541 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:10:44.460304 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 18541
I20260929 19:10:44.460427 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:10:44.460640 10160 external_mini_cluster.cc:1428] Setting key e78ecc81cd81f55106cb6a72e700eb8e
Sep 29 19:10:44 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709044, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:44.468346 18384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:44.458218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:44943 (local address 127.9.236.62:39695)
0929 19:10:44.458292 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:44.458295 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:44.458501 (+ 206us) server_negotiation.cc:407] Connection header received
0929 19:10:44.459117 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:44.459118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:44.459134 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:44.459201 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:44.460344 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:44.460971 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:44.462474 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:44.462689 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:44.462757 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:44.465333 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:44.465335 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:44.465336 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:44.465345 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:44.466769 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:44.467322 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:44.467322 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:44.467323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:44.467381 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:44.467682 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:44.467683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:44.467683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:44.467801 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:44.467863 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:44.468190 (+ 327us) server_negotiation.cc:299] Negotiation successful
0929 19:10:44.468272 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 19:10:44.468683 18672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:44.458159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.2:44943)
0929 19:10:44.458353 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:44.458399 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:44.459004 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:44.459291 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:44.459296 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:44.459939 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:44.460208 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:44.460214 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:44.461069 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:44.461070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:44.462380 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:44.462383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:44.462784 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:44.462785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:44.462932 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:44.463531 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:44.463541 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:44.465209 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:44.466868 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:44.466875 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:44.466878 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:44.467234 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:44.467457 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:44.467458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:44.467458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:44.467604 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:44.467873 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:44.467878 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:44.468049 (+ 171us) client_negotiation.cc:769] Sending connection context
0929 19:10:44.468175 (+ 126us) client_negotiation.cc:240] Negotiation successful
0929 19:10:44.468360 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
WARNING: no policy specified for kudu/127.9.236.3@KRBTEST.COM; defaulting to no policy
I20260929 19:10:44.469385 18670 heartbeater.cc:347] Connected to a master server at 127.9.236.62:39695
I20260929 19:10:44.469494 18670 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:44.469692 18670 heartbeater.cc:511] Master 127.9.236.62:39695 requested a full tablet report, sending...
I20260929 19:10:44.470135 18345 ts_manager.cc:194] Registered new tserver with Master: 7ed7f5b865b74849a2a0f37516a9fffe (127.9.236.2:41947)
I20260929 19:10:44.470579 18345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:44943
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
I20260929 19:10:44.488960 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:39695
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:46543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:44.572161 18675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:44.572369 18675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:44.572391 18675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:44.573590 18675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:44.573642 18675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:44.573668 18675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:10:44.575060 18675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:46543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:39695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:44.575271 18675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:44.575492 18675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:44.580438 18681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:44.580492 18675 server_base.cc:1034] running on GCE node
W20260929 19:10:44.580404 18680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:44.580475 18683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:44.580891 18675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:44.581195 18675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:44.582353 18675 hybrid_clock.cc:648] HybridClock initialized: now 1790709044582345 us; error 66 us; skew 500 ppm
Sep 29 19:10:44 dist-test-slave-wftq krb5kdc[18300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709044, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:44.584944 18675 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:10:44.585500 18675 webserver.cc:460] Webserver started at http://127.9.236.3:38621/ using document root <none> and password file <none>
I20260929 19:10:44.585698 18675 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:44.585744 18675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:44.585855 18675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:10:44.586928 18675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "a22e71aec15a45deafe4a8b7295711e2"
format_stamp: "Formatted at 2026-09-29 19:10:44 on dist-test-slave-wftq"
server_key: "0d059b1e87b1d8b6d6c8e05026a7b993"
server_key_iv: "c065e45cc013f7e28dcd7434b87d4853"
server_key_version: "encryptionkey@0"
I20260929 19:10:44.587260 18675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "a22e71aec15a45deafe4a8b7295711e2"
format_stamp: "Formatted at 2026-09-29 19:10:44 on dist-test-slave-wftq"
server_key: "0d059b1e87b1d8b6d6c8e05026a7b993"
server_key_iv: "c065e45cc013f7e28dcd7434b87d4853"
server_key_version: "encryptionkey@0"
I20260929 19:10:44.588507 18675 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:10:44.589340 18690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:44.589527 18675 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 19:10:44.589619 18675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "a22e71aec15a45deafe4a8b7295711e2"
format_stamp: "Formatted at 2026-09-29 19:10:44 on dist-test-slave-wftq"
server_key: "0d059b1e87b1d8b6d6c8e05026a7b993"
server_key_iv: "c065e45cc013f7e28dcd7434b87d4853"
server_key_version: "encryptionkey@0"
I20260929 19:10:44.589679 18675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:44.605746 18675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:44.607736 18675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:44.607861 18675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:44.608093 18675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:44.608376 18675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:44.608409 18675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:44.608438 18675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:44.608453 18675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:44.614899 18675 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:39463
I20260929 19:10:44.614982 18803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:39463 every 8 connection(s)
I20260929 19:10:44.615286 18675 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
Sep 29 19:10:44 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709044, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:10:44.623855 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 18675
I20260929 19:10:44.623948 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:10:44.624125 10160 external_mini_cluster.cc:1428] Setting key 272fb134ad9bf29cfce2ca7a0c8d93b9
I20260929 19:10:44.624821 18384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:44.616048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:58675 (local address 127.9.236.62:39695)
0929 19:10:44.616134 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:44.616136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:44.616288 (+ 152us) server_negotiation.cc:407] Connection header received
0929 19:10:44.616869 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:44.616871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:44.616884 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:44.616950 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:44.618072 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:44.618574 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:44.619658 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:44.619854 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:44.619912 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:44.622183 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:44.622184 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:44.622185 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:44.622192 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:44.623250 (+ 1058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:44.623627 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:44.623628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:44.623629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:44.623677 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:44.624066 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:44.624068 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:44.624068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:44.624210 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:44.624281 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:44.624605 (+ 324us) server_negotiation.cc:299] Negotiation successful
0929 19:10:44.624702 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 19:10:44.624989 18806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:44.615978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.3:58675)
0929 19:10:44.616147 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:44.616191 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:44.616748 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:44.617029 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:44.617033 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:44.617713 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:44.617956 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:44.617961 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:44.618693 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:44.618694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:44.619577 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:44.619580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:44.619926 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:44.619927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:44.620029 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:44.620545 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:44.620554 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:44.622090 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:10:44.623331 (+ 1241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:44.623337 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:44.623340 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:44.623562 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:44.623784 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:10:44.623785 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:10:44.623786 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:10:44.623939 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:10:44.624293 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:10:44.624300 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:10:44.624455 (+ 155us) client_negotiation.cc:769] Sending connection context
0929 19:10:44.624571 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 19:10:44.624751 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260929 19:10:44.625535 18804 heartbeater.cc:347] Connected to a master server at 127.9.236.62:39695
I20260929 19:10:44.625631 18804 heartbeater.cc:464] Registering TS with master...
I20260929 19:10:44.625810 18804 heartbeater.cc:511] Master 127.9.236.62:39695 requested a full tablet report, sending...
I20260929 19:10:44.626129 18345 ts_manager.cc:194] Registered new tserver with Master: a22e71aec15a45deafe4a8b7295711e2 (127.9.236.3:39463)
I20260929 19:10:44.626502 18345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:58675
I20260929 19:10:44.636236 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.236.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.236.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.236.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.1.keytab.
Entry for principal oryx/127.9.236.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.1.keytab.
WARNING: no policy specified for oryx/127.9.236.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.236.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.236.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.2.keytab.
Entry for principal oryx/127.9.236.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.2.keytab.
WARNING: no policy specified for oryx/127.9.236.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.236.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.236.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.3.keytab.
Entry for principal oryx/127.9.236.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.3.keytab.
I20260929 19:10:44.710314 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 18407
I20260929 19:10:44.714659 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 18541
I20260929 19:10:44.719257 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 18675
I20260929 19:10:44.723555 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 18313
I20260929 19:10:44.728765 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:39695
--webserver_interface=127.9.236.62
--webserver_port=34835
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:46543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:39695
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:10:44.816145 18813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:44.816396 18813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:44.816427 18813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:44.817667 18813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:10:44.817713 18813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:44.817732 18813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:44.817747 18813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:10:44.817762 18813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:10:44.819247 18813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:46543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:39695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:39695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=34835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:44.819509 18813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:44.819739 18813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:44.824908 18818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:44.824885 18821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:44.824949 18813 server_base.cc:1034] running on GCE node
W20260929 19:10:44.824879 18819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:44.825472 18813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:44.825796 18813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:44.826954 18813 hybrid_clock.cc:648] HybridClock initialized: now 1790709044826936 us; error 46 us; skew 500 ppm
Sep 29 19:10:44 dist-test-slave-wftq krb5kdc[18300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709044, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:44.829434 18813 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:10:44.829880 18813 webserver.cc:460] Webserver started at http://127.9.236.62:34835/ using document root <none> and password file <none>
I20260929 19:10:44.830072 18813 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:44.830124 18813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:44.831516 18813 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:44.832296 18828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:44.832471 18813 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:44.832571 18813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "87d34d67728f45de838744a2e87de2f1"
format_stamp: "Formatted at 2026-09-29 19:10:44 on dist-test-slave-wftq"
server_key: "41c1cf8c696a231c8d6ee1d6cd4f033e"
server_key_iv: "f85a47bd3cacfd3860c00d6b228b5ba3"
server_key_version: "encryptionkey@0"
I20260929 19:10:44.832943 18813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:44.849669 18813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:44.851625 18813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:44.851756 18813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:44.855911 18813 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:39695
I20260929 19:10:44.855973 18880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:39695 every 8 connection(s)
I20260929 19:10:44.856320 18813 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:10:44.858084 18881 sys_catalog.cc:263] Verifying existing consensus state
I20260929 19:10:44.858825 18881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1: Bootstrap starting.
I20260929 19:10:44.864194 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 18813
I20260929 19:10:44.864568 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:38825
--local_ip_for_outbound_sockets=127.9.236.1
--tserver_master_addrs=127.9.236.62:39695
--webserver_port=32853
--webserver_interface=127.9.236.1
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:46543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.236.1
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.1.keytab with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:10:44.869565 18881 log.cc:824] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1: Log is configured to *not* fsync() on all Append() calls
I20260929 19:10:44.871407 18881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:10:44.871682 18881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1: Bootstrap complete.
I20260929 19:10:44.874033 18881 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87d34d67728f45de838744a2e87de2f1" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39695 } }
I20260929 19:10:44.874275 18881 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87d34d67728f45de838744a2e87de2f1, State: Initialized, Role: FOLLOWER
I20260929 19:10:44.874442 18881 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87d34d67728f45de838744a2e87de2f1" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39695 } }
I20260929 19:10:44.874511 18881 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:10:44.874552 18881 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:10:44.874598 18881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [term 1 FOLLOWER]: Advancing to term 2
I20260929 19:10:44.875671 18881 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87d34d67728f45de838744a2e87de2f1" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39695 } }
I20260929 19:10:44.875775 18881 leader_election.cc:302] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87d34d67728f45de838744a2e87de2f1; no voters:
I20260929 19:10:44.875972 18881 leader_election.cc:288] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 19:10:44.876022 18885 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 19:10:44.876209 18885 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [term 2 LEADER]: Becoming Leader. State: Replica: 87d34d67728f45de838744a2e87de2f1, State: Running, Role: LEADER
I20260929 19:10:44.876332 18881 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:10:44.876394 18885 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87d34d67728f45de838744a2e87de2f1" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39695 } }
I20260929 19:10:44.876763 18886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "87d34d67728f45de838744a2e87de2f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87d34d67728f45de838744a2e87de2f1" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39695 } } }
I20260929 19:10:44.876857 18886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:44.877291 18887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87d34d67728f45de838744a2e87de2f1. Latest consensus state: current_term: 2 leader_uuid: "87d34d67728f45de838744a2e87de2f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87d34d67728f45de838744a2e87de2f1" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39695 } } }
I20260929 19:10:44.877367 18887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1 [sys.catalog]: This master's current role is: LEADER
I20260929 19:10:44.877537 18891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:10:44.878100 18891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:10:44.878300 18891 catalog_manager.cc:1259] Loaded cluster ID: 8919099b060c423581f9adeb65c89705
I20260929 19:10:44.878578 18891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:10:44.879526 18891 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:10:44.879698 18891 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 87d34d67728f45de838744a2e87de2f1: Loaded TSK: 0
I20260929 19:10:44.879910 18891 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 19:10:44.970366 18883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:44.970582 18883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:44.970614 18883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:44.971774 18883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:44.971817 18883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:44.971853 18883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:10:44.973313 18883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:46543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.1.keytab
--principal=oryx/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:38825
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=32853
--tserver_master_addrs=127.9.236.62:39695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:44.973546 18883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:44.973747 18883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:44.978818 18907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:44.978876 18908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:44.978822 18910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:44.979483 18883 server_base.cc:1034] running on GCE node
I20260929 19:10:44.979667 18883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:44.979861 18883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:44.980995 18883 hybrid_clock.cc:648] HybridClock initialized: now 1790709044980956 us; error 61 us; skew 500 ppm
Sep 29 19:10:44 dist-test-slave-wftq krb5kdc[18300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709044, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:44.983469 18883 init.cc:377] Logged in from keytab as oryx/127.9.236.1@KRBTEST.COM (short username oryx)
I20260929 19:10:44.983908 18883 webserver.cc:460] Webserver started at http://127.9.236.1:32853/ using document root <none> and password file <none>
I20260929 19:10:44.984134 18883 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:44.984192 18883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:44.985486 18883 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:44.986028 18917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:44.986171 18883 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:10:44.986245 18883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "716b8d8217f64c3198ace5654fa9550b"
format_stamp: "Formatted at 2026-09-29 19:10:44 on dist-test-slave-wftq"
server_key: "f7cdabf5d746ca66e81b5372d93f0475"
server_key_iv: "220d3958566a3c3fd23b96168ec6cc40"
server_key_version: "encryptionkey@0"
I20260929 19:10:44.986584 18883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:45.002545 18883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:45.004787 18883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:45.004901 18883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:45.005193 18883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:45.005529 18883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:45.005571 18883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:45.005592 18883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:45.005604 18883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:45.012355 18883 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:38825
I20260929 19:10:45.012393 19030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:38825 every 8 connection(s)
I20260929 19:10:45.012734 18883 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
Sep 29 19:10:45 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:45 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:45.020892 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 18883
I20260929 19:10:45.021202 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:41947
--local_ip_for_outbound_sockets=127.9.236.2
--tserver_master_addrs=127.9.236.62:39695
--webserver_port=44521
--webserver_interface=127.9.236.2
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:46543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.236.2
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.2.keytab with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:10:45.021464 19033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:45.013527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.1:35455)
0929 19:10:45.013698 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:45.013739 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:45.014250 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:45.014648 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:45.014652 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:45.015202 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:45.015376 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:45.015380 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.016460 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.016462 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:45.017286 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:45.017288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.017831 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.017832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:45.017960 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:45.018588 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:45.018597 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:45.021276 (+ 2679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
W20260929 19:10:45.022012 19034 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:45.013671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:35455 (local address 127.9.236.62:39695)
0929 19:10:45.013932 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:45.013941 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:45.013985 (+ 44us) server_negotiation.cc:407] Connection header received
0929 19:10:45.014451 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:45.014455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:45.014494 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:45.014601 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:45.015483 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.016374 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.017356 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.017757 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.017858 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:45.021765 (+ 3907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.1:35455: BlockingRecv error: recv got EOF from 127.9.236.1:35455 (error 108)
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":131,"threads_started":1}
W20260929 19:10:45.022342 19031 heartbeater.cc:644] Failed to heartbeat to 127.9.236.62:39695 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.236.62:39695: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Sep 29 19:10:45 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:45 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:45.028368 19033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:45.022637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.1:41783)
0929 19:10:45.022703 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:45.022714 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:45.022761 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:45.022931 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:45.022932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:45.023118 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:45.023205 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:45.023207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.024078 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.024079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:45.024952 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:45.024955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.025500 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.025501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:45.025578 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:45.026113 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:45.026118 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:45.028293 (+ 2175us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 19:10:45.028625 19034 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:45.022700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:41783 (local address 127.9.236.62:39695)
0929 19:10:45.022765 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:45.022768 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:45.022811 (+ 43us) server_negotiation.cc:407] Connection header received
0929 19:10:45.022832 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:45.022833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:45.022849 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:45.022906 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:45.023296 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.023971 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.025053 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.025378 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.025457 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:45.028561 (+ 3104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.1:41783: BlockingRecv error: recv got EOF from 127.9.236.1:41783 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 29 19:10:45 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:45 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:45.034150 19033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:45.029028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.1:57343)
0929 19:10:45.029167 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:45.029178 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:45.029219 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:45.029404 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:45.029405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:45.029545 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:45.029593 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:45.029595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.030412 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.030413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:45.031260 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:45.031262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.031663 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.031664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:45.031721 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:45.032192 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:45.032196 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:45.034091 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119}
W20260929 19:10:45.034261 19034 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:45.029171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:57343 (local address 127.9.236.62:39695)
0929 19:10:45.029218 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:45.029220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:45.029231 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:10:45.029303 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:45.029304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:45.029317 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:45.029379 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:45.029672 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.030334 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.031344 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.031588 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.031655 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:45.034200 (+ 2545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.1:57343: BlockingRecv error: recv got EOF from 127.9.236.1:57343 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260929 19:10:45.034293 19031 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 19:10:45.101127 19035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:45.101328 19035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:45.101349 19035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:45.102437 19035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:45.102474 19035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:45.102497 19035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:10:45.103883 19035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:46543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.2.keytab
--principal=oryx/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:41947
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=44521
--tserver_master_addrs=127.9.236.62:39695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:45.104086 19035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:45.104292 19035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:45.109304 19043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:45.109463 19035 server_base.cc:1034] running on GCE node
W20260929 19:10:45.109254 19041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:45.109257 19040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:45.109829 19035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:45.110059 19035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:45.111194 19035 hybrid_clock.cc:648] HybridClock initialized: now 1790709045111174 us; error 40 us; skew 500 ppm
Sep 29 19:10:45 dist-test-slave-wftq krb5kdc[18300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709045, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:45.113786 19035 init.cc:377] Logged in from keytab as oryx/127.9.236.2@KRBTEST.COM (short username oryx)
I20260929 19:10:45.114279 19035 webserver.cc:460] Webserver started at http://127.9.236.2:44521/ using document root <none> and password file <none>
I20260929 19:10:45.114490 19035 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:45.114548 19035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:45.115984 19035 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:45.116612 19050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:45.116789 19035 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:45.116878 19035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "7ed7f5b865b74849a2a0f37516a9fffe"
format_stamp: "Formatted at 2026-09-29 19:10:44 on dist-test-slave-wftq"
server_key: "cda4e6abe7abdf7b2ce14058cd2ac1a4"
server_key_iv: "5b3999a94e4879bf3bd60d7a6c2c1884"
server_key_version: "encryptionkey@0"
I20260929 19:10:45.117280 19035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:45.126927 19035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:45.128826 19035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:45.128937 19035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:45.129164 19035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:45.129434 19035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:45.129467 19035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:45.129488 19035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:45.129508 19035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:45.135735 19035 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:41947
I20260929 19:10:45.135836 19163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:41947 every 8 connection(s)
I20260929 19:10:45.136132 19035 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
Sep 29 19:10:45 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:45 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
W20260929 19:10:45.143752 19034 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:45.136874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:55229 (local address 127.9.236.62:39695)
0929 19:10:45.136929 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:45.136932 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:45.137291 (+ 359us) server_negotiation.cc:407] Connection header received
0929 19:10:45.137858 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:45.137859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:45.137873 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:45.137946 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:45.139018 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.139524 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.140561 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.140761 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.140823 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:45.143700 (+ 2877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.2:55229: BlockingRecv error: recv got EOF from 127.9.236.2:55229 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:10:45.143769 19166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:45.136891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.2:55229)
0929 19:10:45.137070 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:45.137153 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:45.137740 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:45.138021 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:45.138025 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:45.138659 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:45.138889 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:45.138893 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.139603 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.139603 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:45.140479 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:45.140482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.140834 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.140835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:45.140934 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:45.141477 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:45.141486 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:45.143601 (+ 2115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":87,"threads_started":1}
W20260929 19:10:45.144167 19164 heartbeater.cc:644] Failed to heartbeat to 127.9.236.62:39695 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.236.62:39695: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
I20260929 19:10:45.146143 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 19035
I20260929 19:10:45.146448 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:39463
--local_ip_for_outbound_sockets=127.9.236.3
--tserver_master_addrs=127.9.236.62:39695
--webserver_port=38621
--webserver_interface=127.9.236.3
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:46543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.236.3
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.3.keytab with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 19:10:45 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:45 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:45.148756 19166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:45.144389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.2:42777)
0929 19:10:45.144432 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:45.144442 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:45.144476 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:45.144637 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:45.144638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:45.144740 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:45.144786 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:45.144787 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.145413 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.145414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:45.145934 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:45.145935 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.146358 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.146359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:45.146412 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:45.146850 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:45.146854 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:45.148681 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260929 19:10:45.148815 19034 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:45.144485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:42777 (local address 127.9.236.62:39695)
0929 19:10:45.144530 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:45.144532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:45.144541 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:45.144564 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:45.144565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:45.144572 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:45.144627 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:45.144841 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.145350 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.146011 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.146282 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.146362 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:45.148767 (+ 2405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.2:42777: BlockingRecv error: recv got EOF from 127.9.236.2:42777 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 29 19:10:45 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:45 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:45.152984 19166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:45.149076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.2:38881)
0929 19:10:45.149152 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:45.149164 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:45.149210 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:45.149327 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:45.149328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:45.149451 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:45.149489 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:45.149491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.150076 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.150077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:45.150576 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:45.150578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.150864 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.150865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:45.150914 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:45.151282 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:45.151287 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:45.152938 (+ 1651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
W20260929 19:10:45.153172 19164 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 19:10:45.153237 19034 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:45.149179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:38881 (local address 127.9.236.62:39695)
0929 19:10:45.149236 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:45.149238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:45.149245 (+ 7us) server_negotiation.cc:407] Connection header received
0929 19:10:45.149262 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:45.149263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:45.149271 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:45.149316 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:45.149567 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.150004 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.150640 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.150812 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.150861 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:45.153160 (+ 2299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.2:38881: BlockingRecv error: recv got EOF from 127.9.236.2:38881 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260929 19:10:45.233359 19167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:10:45.233590 19167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:10:45.233613 19167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:10:45.234776 19167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:10:45.234815 19167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:10:45.234839 19167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:10:45.236362 19167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:46543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/krb5kdc/oryx_127.9.236.3.keytab
--principal=oryx/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:39463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=38621
--tserver_master_addrs=127.9.236.62:39695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:10:45.236604 19167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:10:45.236809 19167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:10:45.241966 19172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:10:45.241986 19173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:45.242086 19167 server_base.cc:1034] running on GCE node
W20260929 19:10:45.241986 19175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:10:45.242393 19167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:10:45.242645 19167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:10:45.243822 19167 hybrid_clock.cc:648] HybridClock initialized: now 1790709045243806 us; error 35 us; skew 500 ppm
Sep 29 19:10:45 dist-test-slave-wftq krb5kdc[18300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709045, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:10:45.250979 19167 init.cc:377] Logged in from keytab as oryx/127.9.236.3@KRBTEST.COM (short username oryx)
I20260929 19:10:45.251518 19167 webserver.cc:460] Webserver started at http://127.9.236.3:38621/ using document root <none> and password file <none>
I20260929 19:10:45.251796 19167 fs_manager.cc:359] Metadata directory not provided
I20260929 19:10:45.251863 19167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:10:45.253381 19167 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:45.254101 19182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:45.254299 19167 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:10:45.254390 19167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "a22e71aec15a45deafe4a8b7295711e2"
format_stamp: "Formatted at 2026-09-29 19:10:44 on dist-test-slave-wftq"
server_key: "0d059b1e87b1d8b6d6c8e05026a7b993"
server_key_iv: "c065e45cc013f7e28dcd7434b87d4853"
server_key_version: "encryptionkey@0"
I20260929 19:10:45.254788 19167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:10:45.267985 19167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:10:45.269878 19167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:10:45.270012 19167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:10:45.270242 19167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:10:45.270550 19167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:10:45.270581 19167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:45.270614 19167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:10:45.270637 19167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:10:45.277560 19167 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:39463
I20260929 19:10:45.277653 19295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:39463 every 8 connection(s)
I20260929 19:10:45.278017 19167 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:10:45.282194 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 19167
Sep 29 19:10:45 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:45 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
W20260929 19:10:45.285678 19034 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:45.278767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:39435 (local address 127.9.236.62:39695)
0929 19:10:45.278876 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:45.278880 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:45.279113 (+ 233us) server_negotiation.cc:407] Connection header received
0929 19:10:45.279664 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:45.279665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:45.279677 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:45.279729 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:45.280708 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.281224 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.282187 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.282404 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.282474 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:45.285614 (+ 3140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.3:39435: BlockingRecv error: recv got EOF from 127.9.236.3:39435 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 19:10:45.285722 19298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:45.278756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.3:39435)
0929 19:10:45.278969 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:45.279019 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:45.279571 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:45.279800 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:45.279805 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:45.280412 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:45.280621 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:45.280625 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.281302 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.281303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:45.282105 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:45.282107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.282490 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.282492 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:45.282669 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:45.283239 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:45.283248 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:45.285538 (+ 2290us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":92,"threads_started":1}
W20260929 19:10:45.286190 19296 heartbeater.cc:644] Failed to heartbeat to 127.9.236.62:39695 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.236.62:39695: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
I20260929 19:10:45.289849 19300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:45.282748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45976 (local address 127.9.236.62:39695)
0929 19:10:45.282891 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:45.282892 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:45.282900 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:10:45.282918 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:45.282919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:45.282927 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:45.282980 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:45.283566 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.284020 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.284760 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.284989 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.285039 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:45.286425 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:10:45.286429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:10:45.286430 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:10:45.286442 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:10:45.288173 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:45.288540 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:45.288543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:45.288544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:45.288627 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:10:45.288855 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:10:45.288856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:10:45.288856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:10:45.289158 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:10:45.289285 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 19:10:45.289629 (+ 344us) server_negotiation.cc:299] Negotiation successful
0929 19:10:45.289768 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
Sep 29 19:10:45 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:45 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:45.291308 19298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:45.286422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.3:50483)
0929 19:10:45.286536 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:45.286549 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:45.286603 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:45.286748 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:45.286748 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:45.286867 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:45.286930 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:45.286933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.287678 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.287679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:45.288421 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:45.288424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.288784 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.288785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:45.288850 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:45.289323 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:45.289328 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:45.291260 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90}
W20260929 19:10:45.291411 19034 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:45.286475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:50483 (local address 127.9.236.62:39695)
0929 19:10:45.286522 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:45.286524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:45.286589 (+ 65us) server_negotiation.cc:407] Connection header received
0929 19:10:45.286673 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:45.286674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:45.286683 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:45.286739 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:45.287013 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.287591 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.288502 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.288711 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.288778 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:45.291355 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.3:50483: BlockingRecv error: recv got EOF from 127.9.236.3:50483 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Sep 29 19:10:45 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:45 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:45.295297 19298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:45.291694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.3:36597)
0929 19:10:45.291747 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:45.291758 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:45.291801 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:45.291906 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:45.291907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:45.292002 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:45.292025 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:45.292026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.292599 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.292600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:45.293134 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:45.293136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.293458 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.293459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:45.293493 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:45.293792 (+ 299us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:45.293796 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:45.295252 (+ 1456us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260929 19:10:45.295409 19034 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:45.291765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:36597 (local address 127.9.236.62:39695)
0929 19:10:45.291821 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:45.291822 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:45.291832 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:10:45.291849 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:45.291849 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:45.291857 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:45.291902 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:45.292078 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.292512 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.293199 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:45.293397 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:45.293450 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:45.295341 (+ 1891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.3:36597: BlockingRecv error: recv got EOF from 127.9.236.3:36597 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260929 19:10:45.295423 19296 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 29 19:10:46 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:46 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:46.039664 19301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:46.034769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.1:59657)
0929 19:10:46.034917 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:46.034930 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:46.034993 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:46.035158 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:46.035159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:46.035366 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:46.035436 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:46.035438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:46.036115 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:46.036115 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:46.036739 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:46.036741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:46.037140 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:46.037142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:46.037200 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:46.037704 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:46.037709 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:46.039590 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
W20260929 19:10:46.039808 19302 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:46.034842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:59657 (local address 127.9.236.62:39695)
0929 19:10:46.034965 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:46.034967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:46.034978 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:10:46.035052 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:46.035053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:46.035063 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:46.035137 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:46.035499 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:46.036043 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:46.036803 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:46.036991 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:46.037075 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:46.039709 (+ 2634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.1:59657: BlockingRecv error: recv got EOF from 127.9.236.1:59657 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
Sep 29 19:10:46 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:46 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:46.158365 19303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:46.153593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.2:49107)
0929 19:10:46.153741 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:46.153754 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:46.153828 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:46.153964 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:46.153965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:46.154157 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:46.154224 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:46.154226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:46.154884 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:46.154885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:46.155494 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:46.155496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:46.155831 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:46.155832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:46.155877 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:46.156317 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:46.156322 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:46.158266 (+ 1944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":60,"threads_started":1}
W20260929 19:10:46.158416 19302 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:46.153657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:49107 (local address 127.9.236.62:39695)
0929 19:10:46.153747 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:46.153750 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:46.153825 (+ 75us) server_negotiation.cc:407] Connection header received
0929 19:10:46.153891 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:46.153892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:46.153904 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:46.153948 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:46.154290 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:46.154805 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:46.155578 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:46.155764 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:46.155822 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:46.158360 (+ 2538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.2:49107: BlockingRecv error: recv got EOF from 127.9.236.2:49107 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 29 19:10:46 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:46 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:46.300478 19304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:46.295889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.3:41893)
0929 19:10:46.296068 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:46.296084 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:46.296160 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:46.296302 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:46.296303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:46.296477 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:46.296526 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:46.296528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:46.297207 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:46.297208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:46.297758 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:46.297760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:46.298070 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:46.298071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:46.298121 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:46.298565 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:46.298569 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:46.300378 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":78,"threads_started":1}
W20260929 19:10:46.300532 19302 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:46.296027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:41893 (local address 127.9.236.62:39695)
0929 19:10:46.296128 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:46.296130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:46.296139 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:46.296227 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:46.296228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:46.296242 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:46.296288 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:46.296598 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:46.297080 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:46.297836 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:46.298014 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:46.298066 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:46.300469 (+ 2403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.3:41893: BlockingRecv error: recv got EOF from 127.9.236.3:41893 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 29 19:10:47 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:47 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:47.045411 19305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:47.040229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.1:55355)
0929 19:10:47.040379 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:47.040396 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:47.040545 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:47.040714 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:47.040715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:47.040905 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:47.040971 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:47.040973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:47.041733 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:47.041734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:47.042468 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:47.042471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:47.042957 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:47.042958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:47.043045 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:47.043520 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:47.043525 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:47.045344 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":78,"threads_started":1}
W20260929 19:10:47.045547 19306 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:47.040318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:55355 (local address 127.9.236.62:39695)
0929 19:10:47.040449 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:47.040451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:47.040551 (+ 100us) server_negotiation.cc:407] Connection header received
0929 19:10:47.040628 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:47.040629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:47.040639 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:47.040695 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:47.041139 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:47.041644 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:47.042588 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:47.042858 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:47.042914 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:47.045457 (+ 2543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.1:55355: BlockingRecv error: recv got EOF from 127.9.236.1:55355 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
Sep 29 19:10:47 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:47 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:47.163729 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:47.158936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.2:33357)
0929 19:10:47.159069 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:47.159083 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:47.159151 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:47.159290 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:47.159291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:47.159479 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:47.159547 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:47.159548 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:47.160175 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:47.160176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:47.160771 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:47.160773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:47.161189 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:47.161191 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:47.161256 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:47.161781 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:47.161786 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:47.163650 (+ 1864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
W20260929 19:10:47.163807 19306 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:47.159083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:33357 (local address 127.9.236.62:39695)
0929 19:10:47.159141 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:47.159143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:47.159155 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:10:47.159220 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:47.159221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:47.159232 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:47.159274 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:47.159609 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:47.160112 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:47.160853 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:47.161053 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:47.161148 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:47.163747 (+ 2599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.2:33357: BlockingRecv error: recv got EOF from 127.9.236.2:33357 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 29 19:10:47 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:47 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:47.306043 19308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:47.301046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.3:36477)
0929 19:10:47.301292 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:47.301305 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:47.301390 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:47.301543 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:47.301543 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:47.301718 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:47.301768 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:47.301770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:47.302436 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:47.302436 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:47.302984 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:47.302985 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:47.303397 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:47.303398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:47.303436 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:47.303919 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:47.303923 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:47.305975 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":157,"threads_started":1}
W20260929 19:10:47.306252 19306 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:47.301214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:36477 (local address 127.9.236.62:39695)
0929 19:10:47.301301 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:47.301304 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:47.301380 (+ 76us) server_negotiation.cc:407] Connection header received
0929 19:10:47.301444 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:47.301445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:47.301459 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:47.301523 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:47.301829 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:47.302373 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:47.303084 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:47.303318 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:47.303400 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:47.306085 (+ 2685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.3:36477: BlockingRecv error: recv got EOF from 127.9.236.3:36477 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 29 19:10:48 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:48 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:48.050839 19309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:48.045971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.1:57119)
0929 19:10:48.046136 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:48.046152 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:48.046233 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:48.046369 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:48.046370 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:48.046560 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:48.046626 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:48.046628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:48.047262 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:48.047263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:48.047870 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:48.047872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:48.048222 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:48.048222 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:48.048261 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:48.048767 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:48.048771 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:48.050740 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
W20260929 19:10:48.050938 19310 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:48.046075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:57119 (local address 127.9.236.62:39695)
0929 19:10:48.046251 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:48.046254 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:48.046263 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:48.046285 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:48.046286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:48.046296 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:48.046360 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:48.046693 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:48.047199 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:48.047961 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:48.048158 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:48.048214 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:48.050859 (+ 2645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.1:57119: BlockingRecv error: recv got EOF from 127.9.236.1:57119 (error 108)
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":103,"threads_started":1}
Sep 29 19:10:48 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:48 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:48.169538 19311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:48.164272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.2:53809)
0929 19:10:48.164467 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:48.164485 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:48.164582 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:48.164753 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:48.164754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:48.164972 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:48.165044 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:48.165046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:48.165836 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:48.165838 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:48.166537 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:48.166540 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:48.166860 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:48.166861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:48.166914 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:48.167471 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:48.167483 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:48.169442 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":79,"threads_started":1}
W20260929 19:10:48.169584 19310 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:48.164408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:53809 (local address 127.9.236.62:39695)
0929 19:10:48.164488 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:48.164490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:48.164571 (+ 81us) server_negotiation.cc:407] Connection header received
0929 19:10:48.164669 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:48.164670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:48.164681 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:48.164726 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:48.165137 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:48.165727 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:48.166607 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:48.166799 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:48.166852 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:48.169531 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.2:53809: BlockingRecv error: recv got EOF from 127.9.236.2:53809 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 29 19:10:48 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:48 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:48.311841 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:48.306642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.3:46249)
0929 19:10:48.306824 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:48.306837 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:48.306942 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:48.307118 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:48.307119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:48.307311 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:48.307377 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:48.307379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:48.308063 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:48.308064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:48.308862 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:48.308865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:48.309287 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:48.309288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:48.309350 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:48.309874 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:48.309879 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:48.311751 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":74,"threads_started":1}
W20260929 19:10:48.311910 19310 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:48.306741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:46249 (local address 127.9.236.62:39695)
0929 19:10:48.306792 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:48.306794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:48.306936 (+ 142us) server_negotiation.cc:407] Connection header received
0929 19:10:48.307023 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:48.307024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:48.307038 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:48.307094 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:48.307447 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:48.307952 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:48.308944 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:48.309200 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:48.309271 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:48.311846 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.3:46249: BlockingRecv error: recv got EOF from 127.9.236.3:46249 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Sep 29 19:10:49 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:49 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:49.056396 19313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:49.051426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.1:38747)
0929 19:10:49.051571 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:49.051582 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:49.051644 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:49.051795 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:49.051796 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:49.051986 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:49.052056 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:49.052058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:49.052794 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:49.052795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:49.053425 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:49.053427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:49.053790 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:49.053791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:49.053841 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:49.054352 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:49.054356 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:49.056322 (+ 1966us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
W20260929 19:10:49.056527 19314 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:49.051514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:38747 (local address 127.9.236.62:39695)
0929 19:10:49.051664 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:49.051665 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:49.051674 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:49.051695 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:49.051696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:49.051707 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:49.051774 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:49.052170 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:49.052688 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:49.053514 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:49.053708 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:49.053768 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:49.056444 (+ 2676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.1:38747: BlockingRecv error: recv got EOF from 127.9.236.1:38747 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
Sep 29 19:10:49 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:49 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:49.174942 19315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:49.170090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.2:33237)
0929 19:10:49.170256 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:49.170266 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:49.170328 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:49.170478 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:49.170479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:49.170686 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:49.170746 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:49.170748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:49.171546 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:49.171547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:49.172104 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:49.172105 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:49.172441 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:49.172442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:49.172480 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:49.172932 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:49.172975 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:49.174868 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":83,"threads_started":1}
W20260929 19:10:49.175040 19314 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:49.170215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:33237 (local address 127.9.236.62:39695)
0929 19:10:49.170269 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:49.170272 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:49.170317 (+ 45us) server_negotiation.cc:407] Connection header received
0929 19:10:49.170382 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:49.170382 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:49.170395 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:49.170453 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:49.170901 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:49.171464 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:49.172177 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:49.172380 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:49.172431 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:49.174964 (+ 2533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.2:33237: BlockingRecv error: recv got EOF from 127.9.236.2:33237 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Sep 29 19:10:49 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:49 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:49.317330 19316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:49.312432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.3:37303)
0929 19:10:49.312616 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:49.312629 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:49.312698 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:49.312869 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:49.312869 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:49.313046 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:49.313152 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:49.313154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:49.313835 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:49.313836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:49.314420 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:49.314422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:49.314746 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:49.314746 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:49.314786 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:49.315228 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:49.315234 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:49.317247 (+ 2013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":106,"threads_started":1}
W20260929 19:10:49.317395 19314 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:49.312533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:37303 (local address 127.9.236.62:39695)
0929 19:10:49.312628 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:49.312630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:49.312699 (+ 69us) server_negotiation.cc:407] Connection header received
0929 19:10:49.312768 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:49.312769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:49.312782 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:49.312861 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:49.313253 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:49.313750 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:49.314500 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:49.314685 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:49.314739 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:49.317338 (+ 2599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.3:37303: BlockingRecv error: recv got EOF from 127.9.236.3:37303 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Sep 29 19:10:50 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:50 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:50.061839 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:50.056929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.1:37243)
0929 19:10:50.057161 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:50.057176 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:50.057245 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:50.057435 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:50.057436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:50.057642 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:50.057704 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:50.057706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:50.058427 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:50.058427 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:50.059038 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:50.059040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:50.059382 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:50.059383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:50.059423 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:50.059871 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:50.059875 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:50.061775 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":90,"threads_started":1}
W20260929 19:10:50.061944 19318 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:50.057042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:37243 (local address 127.9.236.62:39695)
0929 19:10:50.057216 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:50.057218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:50.057227 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:50.057298 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:50.057298 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:50.057310 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:50.057386 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:50.057781 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:50.058361 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:50.059112 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:50.059310 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:50.059376 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:50.061876 (+ 2500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.1:37243: BlockingRecv error: recv got EOF from 127.9.236.1:37243 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
Sep 29 19:10:50 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:50 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:50.179931 19319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:50.175456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.2:50567)
0929 19:10:50.175598 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:50.175609 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:50.175671 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:50.175803 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:50.175804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:50.176004 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:50.176056 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:50.176058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:50.176683 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:50.176684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:50.177394 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:50.177397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:50.177699 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:50.177700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:50.177750 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:50.178194 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:50.178199 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:50.179842 (+ 1643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":64,"threads_started":1}
W20260929 19:10:50.179988 19318 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:50.175618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:50567 (local address 127.9.236.62:39695)
0929 19:10:50.175691 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:50.175694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:50.175703 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:50.175732 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:50.175733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:50.175745 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:50.175791 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:50.176118 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:50.176614 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:50.177471 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:50.177647 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:50.177697 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:50.179934 (+ 2237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.2:50567: BlockingRecv error: recv got EOF from 127.9.236.2:50567 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 29 19:10:50 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:50 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:50.325508 19320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:50.317853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.3:36975)
0929 19:10:50.318003 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:50.318014 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:50.318076 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:50.318224 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:50.318224 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:50.318376 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:50.318429 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:50.318431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:50.319132 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:50.319133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:50.319731 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:50.319733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:50.320102 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:50.320103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:50.320150 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:50.323579 (+ 3429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:50.323584 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:50.325448 (+ 1864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
W20260929 19:10:50.325632 19318 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:50.317932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:36975 (local address 127.9.236.62:39695)
0929 19:10:50.317995 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:50.317997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:50.318071 (+ 74us) server_negotiation.cc:407] Connection header received
0929 19:10:50.318142 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:50.318143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:50.318158 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:50.318208 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:50.318488 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:50.319035 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:50.319831 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:50.320022 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:50.320087 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:50.325556 (+ 5469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.3:36975: BlockingRecv error: recv got EOF from 127.9.236.3:36975 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 29 19:10:51 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:51 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:51.067323 19321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:51.062443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.1:41237)
0929 19:10:51.062616 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:51.062648 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:51.062753 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:51.062914 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:51.062915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:51.063110 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:51.063181 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:51.063183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:51.063823 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:51.063823 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:51.064442 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:51.064444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:51.064840 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:51.064841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:51.064907 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:51.065450 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:51.065455 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:51.067254 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":80,"threads_started":1}
W20260929 19:10:51.067494 19322 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:51.062578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:41237 (local address 127.9.236.62:39695)
0929 19:10:51.062759 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:51.062761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:51.062797 (+ 36us) server_negotiation.cc:407] Connection header received
0929 19:10:51.062820 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:51.062821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:51.062831 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:51.062898 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:51.063249 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:51.063756 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:51.064522 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:51.064766 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:51.064829 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:51.067401 (+ 2572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.1:41237: BlockingRecv error: recv got EOF from 127.9.236.1:41237 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":97,"threads_started":1}
Sep 29 19:10:51 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:51 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:51.185218 19323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:51.180478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.2:38049)
0929 19:10:51.180671 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:51.180686 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:51.180766 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:51.180900 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:51.180901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:51.181090 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:51.181174 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:51.181176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:51.181823 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:51.181829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:51.182347 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:51.182348 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:51.182660 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:51.182661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:51.182710 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:51.183152 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:51.183157 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:51.185142 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":85,"threads_started":1}
W20260929 19:10:51.185267 19322 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:51.180562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:38049 (local address 127.9.236.62:39695)
0929 19:10:51.180665 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:51.180667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:51.180754 (+ 87us) server_negotiation.cc:407] Connection header received
0929 19:10:51.180824 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:51.180825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:51.180837 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:51.180889 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:51.181243 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:51.181750 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:51.182419 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:51.182604 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:51.182656 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:51.185217 (+ 2561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.2:38049: BlockingRecv error: recv got EOF from 127.9.236.2:38049 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 29 19:10:51 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:51 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:51.332104 19324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:51.326049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.3:60361)
0929 19:10:51.326273 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:51.326286 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:51.326375 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:51.326562 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:51.326563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:51.326729 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:51.326780 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:51.326782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:51.327521 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:51.327522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:51.328391 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:51.328394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:51.328857 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:51.328858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:51.328962 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:51.329579 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:51.329584 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:51.332011 (+ 2427us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":124,"threads_started":1}
W20260929 19:10:51.332343 19322 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:51.326135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:60361 (local address 127.9.236.62:39695)
0929 19:10:51.326242 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:51.326246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:51.326374 (+ 128us) server_negotiation.cc:407] Connection header received
0929 19:10:51.326427 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:51.326428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:51.326442 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:51.326497 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:51.326860 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:51.327434 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:51.328482 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:51.328781 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:51.328843 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:51.332273 (+ 3430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.3:60361: BlockingRecv error: recv got EOF from 127.9.236.3:60361 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 29 19:10:52 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:52 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:52.072742 19325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:52.067825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.1:34005)
0929 19:10:52.067993 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:52.068004 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:52.068061 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:52.068252 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:52.068252 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:52.068417 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:52.068473 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:52.068475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:52.069266 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:52.069267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:52.069832 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:52.069834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:52.070203 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:52.070204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:52.070260 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:52.070759 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:52.070763 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:52.072647 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
W20260929 19:10:52.072830 19326 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:52.067931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:34005 (local address 127.9.236.62:39695)
0929 19:10:52.068103 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:52.068105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:52.068115 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:10:52.068145 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:52.068146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:52.068160 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:52.068236 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:52.068575 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:52.069179 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:52.069909 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:52.070107 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:52.070170 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:52.072762 (+ 2592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.1:34005: BlockingRecv error: recv got EOF from 127.9.236.1:34005 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":91,"threads_started":1}
Sep 29 19:10:52 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:52 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:52.191138 19327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:52.185773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.2:60699)
0929 19:10:52.185974 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:52.185991 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:52.186130 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:52.186291 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:52.186292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:52.186502 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:52.186557 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:52.186559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:52.187334 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:52.187336 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:52.187943 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:52.187946 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:52.188261 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:52.188262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:52.188301 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:52.188819 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:52.188824 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:52.191052 (+ 2228us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":101,"threads_started":1}
W20260929 19:10:52.191202 19326 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:52.185906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:60699 (local address 127.9.236.62:39695)
0929 19:10:52.185997 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:52.185999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:52.186118 (+ 119us) server_negotiation.cc:407] Connection header received
0929 19:10:52.186221 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:52.186222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:52.186233 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:52.186279 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:52.186664 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:52.187242 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:52.188017 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:52.188200 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:52.188251 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:52.191147 (+ 2896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.2:60699: BlockingRecv error: recv got EOF from 127.9.236.2:60699 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 29 19:10:52 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:52 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:52.337248 19328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:52.332578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.3:43911)
0929 19:10:52.332704 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:52.332714 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:52.332773 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:52.332915 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:52.332915 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:52.333070 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:52.333153 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:52.333155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:52.333824 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:52.333825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:52.334368 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:52.334369 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:52.334723 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:52.334724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:52.334764 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:52.335236 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:52.335239 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:52.337186 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
W20260929 19:10:52.337373 19326 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:52.332655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:43911 (local address 127.9.236.62:39695)
0929 19:10:52.332720 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:52.332722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:52.332763 (+ 41us) server_negotiation.cc:407] Connection header received
0929 19:10:52.332825 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:52.332826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:52.332842 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:52.332900 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:52.333213 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:52.333757 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:52.334452 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:52.334659 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:52.334716 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:52.337287 (+ 2571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.3:43911: BlockingRecv error: recv got EOF from 127.9.236.3:43911 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 29 19:10:53 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:53 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:53.078864 19329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:53.073430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.1:56065)
0929 19:10:53.073575 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:53.073586 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:53.073635 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:53.073867 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:53.073867 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:53.074025 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:53.074077 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:53.074079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:53.074844 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:53.074845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:53.075386 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:53.075388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:53.075852 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:53.075853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:53.075940 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:53.076462 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:53.076466 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:53.078767 (+ 2301us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":65,"threads_started":1}
W20260929 19:10:53.078958 19330 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:53.073581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:56065 (local address 127.9.236.62:39695)
0929 19:10:53.073732 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:53.073734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:53.073742 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:10:53.073766 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:53.073766 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:53.073777 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:53.073850 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:53.074182 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:53.074746 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:53.075491 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:53.075750 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:53.075836 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:53.078878 (+ 3042us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.1:56065: BlockingRecv error: recv got EOF from 127.9.236.1:56065 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":66,"threads_started":1}
Sep 29 19:10:53 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:53 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:53.196518 19331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:53.191641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.2:34759)
0929 19:10:53.191770 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:53.191780 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:53.191876 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:53.192024 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:53.192024 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:53.192181 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:53.192236 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:53.192238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:53.193083 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:53.193084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:53.193685 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:53.193687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:53.194064 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:53.194065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:53.194119 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:53.194591 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:53.194595 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:53.196421 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":57,"threads_started":1}
W20260929 19:10:53.196589 19330 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:53.191700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:34759 (local address 127.9.236.62:39695)
0929 19:10:53.191767 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:53.191769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:53.191828 (+ 59us) server_negotiation.cc:407] Connection header received
0929 19:10:53.191936 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:53.191937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:53.191949 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:53.192006 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:53.192300 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:53.193003 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:53.193758 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:53.193969 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:53.194029 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:53.196528 (+ 2499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.2:34759: BlockingRecv error: recv got EOF from 127.9.236.2:34759 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 29 19:10:53 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:53 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:53.342482 19332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:53.337786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.3:43547)
0929 19:10:53.337952 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:53.337963 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:53.338022 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:53.338174 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:53.338175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:53.338351 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:53.338431 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:53.338432 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:53.339086 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:53.339087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:53.339715 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:53.339717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:53.340075 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:53.340075 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:53.340152 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:53.340616 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:53.340621 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:53.342412 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":92,"threads_started":1}
W20260929 19:10:53.342653 19330 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:53.337941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:43547 (local address 127.9.236.62:39695)
0929 19:10:53.338003 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:53.338005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:53.338017 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:10:53.338078 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:53.338079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:53.338091 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:53.338156 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:53.338491 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:53.338987 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:53.339810 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:53.340007 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:53.340069 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:53.342559 (+ 2490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.3:43547: BlockingRecv error: recv got EOF from 127.9.236.3:43547 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 29 19:10:54 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:54 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:54.084244 19333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:54.079448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.1:51081)
0929 19:10:54.079587 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:54.079597 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:54.079647 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:54.079871 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:54.079871 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:54.080034 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:54.080085 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:54.080087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:54.080887 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:54.080888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:54.081481 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:54.081483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:54.081827 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:54.081827 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:54.081866 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:54.082319 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:54.082323 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:54.084182 (+ 1859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":62,"threads_started":1}
W20260929 19:10:54.084367 19334 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:54.079565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:51081 (local address 127.9.236.62:39695)
0929 19:10:54.079715 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:54.079717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:54.079726 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:10:54.079752 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:54.079753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:54.079765 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:54.079849 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:54.080191 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:54.080759 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:54.081560 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:54.081764 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:54.081818 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:54.084284 (+ 2466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.1:51081: BlockingRecv error: recv got EOF from 127.9.236.1:51081 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
Sep 29 19:10:54 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:54 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:54.201996 19335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:54.197111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.2:33287)
0929 19:10:54.197230 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:54.197239 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:54.197284 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:54.197518 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:54.197520 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:54.197684 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:54.197731 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:54.197733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:54.198525 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:54.198526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:54.199068 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:54.199070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:54.199467 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:54.199468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:54.199527 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:54.200061 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:54.200065 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:54.201936 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
W20260929 19:10:54.202203 19334 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:54.197215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:33287 (local address 127.9.236.62:39695)
0929 19:10:54.197274 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:54.197276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:54.197287 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:10:54.197344 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:54.197344 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:54.197357 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:54.197424 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:54.197836 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:54.198427 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:54.199174 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:54.199371 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:54.199437 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:54.202088 (+ 2651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.2:33287: BlockingRecv error: recv got EOF from 127.9.236.2:33287 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Sep 29 19:10:54 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:54 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:54.347978 19336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:54.343044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.3:42835)
0929 19:10:54.343250 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:54.343264 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:54.343349 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:54.343505 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:54.343506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:54.343669 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:54.343728 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:54.343730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:54.344417 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:54.344418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:54.344960 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:54.344961 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:54.345353 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:54.345354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:54.345421 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:54.345890 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:54.345895 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:54.347909 (+ 2014us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":91,"threads_started":1}
W20260929 19:10:54.348078 19334 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:54.343163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:42835 (local address 127.9.236.62:39695)
0929 19:10:54.343250 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:54.343252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:54.343342 (+ 90us) server_negotiation.cc:407] Connection header received
0929 19:10:54.343416 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:54.343418 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:54.343431 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:54.343486 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:54.343787 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:54.344337 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:54.345042 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:54.345273 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:54.345344 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:54.348005 (+ 2661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.3:42835: BlockingRecv error: recv got EOF from 127.9.236.3:42835 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 19:10:55 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:55 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:55.089810 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:55.084764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.1:40671)
0929 19:10:55.084911 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:55.084927 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:55.085001 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:55.085224 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:55.085225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:55.085419 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:55.085485 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:55.085486 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:55.086199 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:55.086200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:55.086782 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:55.086784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:55.087174 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:55.087175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:55.087223 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:55.087757 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:55.087762 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:55.089709 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
W20260929 19:10:55.089857 19338 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:55.084847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:40671 (local address 127.9.236.62:39695)
0929 19:10:55.085059 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:55.085062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:55.085073 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:10:55.085095 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:55.085095 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:55.085130 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:55.085207 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:55.085554 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:55.086097 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:55.086889 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:55.087088 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:55.087150 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:55.089793 (+ 2643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.1:40671: BlockingRecv error: recv got EOF from 127.9.236.1:40671 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":117,"threads_started":1}
Sep 29 19:10:55 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:55 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:55.207348 19339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:55.202597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.2:40793)
0929 19:10:55.202762 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:55.202774 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:55.202850 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:55.202994 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:55.202995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:55.203162 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:55.203209 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:55.203210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:55.204010 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:55.204010 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:55.204552 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:55.204553 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:55.204871 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:55.204872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:55.204909 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:55.205412 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:55.205416 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:55.207272 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":80,"threads_started":1}
W20260929 19:10:55.207466 19338 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:55.202708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:40793 (local address 127.9.236.62:39695)
0929 19:10:55.202776 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:55.202778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:55.202832 (+ 54us) server_negotiation.cc:407] Connection header received
0929 19:10:55.202925 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:55.202926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:55.202936 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:55.202978 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:55.203311 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:55.203934 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:55.204621 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:55.204810 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:55.204862 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:55.207390 (+ 2528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.2:40793: BlockingRecv error: recv got EOF from 127.9.236.2:40793 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 29 19:10:55 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:55 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:55.359193 19340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:55.348526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.3:60587)
0929 19:10:55.348666 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:55.348676 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:55.348728 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:55.348876 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:55.348877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:55.349048 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:55.349093 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:55.349094 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:55.349748 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:55.349749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:55.350356 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:55.350359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:55.351460 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:55.351461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:55.351524 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:55.356976 (+ 5452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:55.356983 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:55.359109 (+ 2126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":61,"threads_started":1}
W20260929 19:10:55.359295 19338 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:55.348675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:60587 (local address 127.9.236.62:39695)
0929 19:10:55.348733 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:55.348735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:55.348780 (+ 45us) server_negotiation.cc:407] Connection header received
0929 19:10:55.348802 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:55.348803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:55.348816 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:55.348859 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:55.349169 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:55.349673 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:55.350453 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:55.350813 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:55.350896 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:55.359229 (+ 8333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.3:60587: BlockingRecv error: recv got EOF from 127.9.236.3:60587 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 29 19:10:56 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:56 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:56.095232 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:56.090391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.1:36195)
0929 19:10:56.090544 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:56.090555 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:56.090619 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:56.090776 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:56.090776 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:56.090969 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:56.091100 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:56.091102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:56.091780 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:56.091781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:56.092381 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:56.092383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:56.092763 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:56.092763 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:56.092807 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:56.093358 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:56.093363 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:56.095148 (+ 1785us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
W20260929 19:10:56.095345 19342 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:56.090462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:36195 (local address 127.9.236.62:39695)
0929 19:10:56.090587 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:56.090589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:56.090602 (+ 13us) server_negotiation.cc:407] Connection header received
0929 19:10:56.090671 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:56.090672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:56.090683 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:56.090759 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:56.091165 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:56.091702 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:56.092489 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:56.092690 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:56.092760 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:56.095266 (+ 2506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.1:36195: BlockingRecv error: recv got EOF from 127.9.236.1:36195 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
Sep 29 19:10:56 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:56 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:56.212836 19343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:56.207872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.2:49665)
0929 19:10:56.208040 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:56.208054 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:56.208142 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:56.208282 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:56.208282 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:56.208460 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:56.208510 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:56.208511 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:56.209319 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:56.209320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:56.209888 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:56.209890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:56.210253 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:56.210253 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:56.210312 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:56.210868 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:56.210873 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:56.212756 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":96,"threads_started":1}
W20260929 19:10:56.212944 19342 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:56.207980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:49665 (local address 127.9.236.62:39695)
0929 19:10:56.208086 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:56.208089 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:56.208126 (+ 37us) server_negotiation.cc:407] Connection header received
0929 19:10:56.208207 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:56.208208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:56.208221 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:56.208273 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:56.208611 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:56.209200 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:56.209979 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:56.210185 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:56.210244 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:56.212867 (+ 2623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.2:49665: BlockingRecv error: recv got EOF from 127.9.236.2:49665 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 29 19:10:56 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:56 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:56.364388 19344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:56.359687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.3:42275)
0929 19:10:56.359830 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:56.359842 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:56.359912 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:56.360059 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:56.360060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:56.360275 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:56.360339 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:56.360341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:56.361061 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:56.361062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:56.361688 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:56.361690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:56.362014 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:56.362014 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:56.362052 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:56.362510 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:56.362514 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:56.364303 (+ 1789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
W20260929 19:10:56.364457 19342 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:56.359784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:42275 (local address 127.9.236.62:39695)
0929 19:10:56.359838 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:56.359840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:56.359899 (+ 59us) server_negotiation.cc:407] Connection header received
0929 19:10:56.359978 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:56.359979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:56.359991 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:56.360041 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:56.360452 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:56.360976 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:56.361756 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:56.361951 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:56.362011 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:56.364403 (+ 2392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.3:42275: BlockingRecv error: recv got EOF from 127.9.236.3:42275 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Sep 29 19:10:57 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:57 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:57.100800 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:57.095784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.1:55695)
0929 19:10:57.095960 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:57.095971 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:57.096027 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:57.096221 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:57.096222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:57.096427 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:57.096488 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:57.096490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:57.097205 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:57.097206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:57.097789 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:57.097791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:57.098182 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:57.098183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:57.098239 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:57.098716 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:57.098720 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:57.100697 (+ 1977us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":104,"threads_started":1}
W20260929 19:10:57.100860 19346 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:57.095893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:55695 (local address 127.9.236.62:39695)
0929 19:10:57.096063 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:57.096065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:57.096084 (+ 19us) server_negotiation.cc:407] Connection header received
0929 19:10:57.096110 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:57.096110 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:57.096122 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:57.096201 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:57.096583 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:57.097125 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:57.097878 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:57.098082 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:57.098146 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:57.100801 (+ 2655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.1:55695: BlockingRecv error: recv got EOF from 127.9.236.1:55695 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":71,"threads_started":1}
Sep 29 19:10:57 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:57 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:57.218390 19347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:57.213418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.2:56429)
0929 19:10:57.213568 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:57.213581 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:57.213664 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:57.213843 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:57.213844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:57.214051 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:57.214116 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:57.214118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:57.214750 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:57.214751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:57.215381 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:57.215383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:57.215760 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:57.215761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:57.215804 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:57.216358 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:57.216363 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:57.218302 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":68,"threads_started":1}
W20260929 19:10:57.218463 19346 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:57.213526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:56429 (local address 127.9.236.62:39695)
0929 19:10:57.213590 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:57.213592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:57.213665 (+ 73us) server_negotiation.cc:407] Connection header received
0929 19:10:57.213718 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:57.213719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:57.213729 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:57.213795 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:57.214188 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:57.214678 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:57.215477 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:57.215684 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:57.215791 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:57.218408 (+ 2617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.2:56429: BlockingRecv error: recv got EOF from 127.9.236.2:56429 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 29 19:10:57 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:57 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:57.369858 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:57.364905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.3:52829)
0929 19:10:57.365127 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:57.365140 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:57.365222 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:57.365364 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:57.365365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:57.365531 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:57.365584 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:57.365586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:57.366339 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:57.366340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:57.366887 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:57.366888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:57.367282 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:57.367282 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:57.367335 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:57.367835 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:57.367839 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:57.369770 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":86,"threads_started":1}
W20260929 19:10:57.369951 19346 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:57.365017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:52829 (local address 127.9.236.62:39695)
0929 19:10:57.365131 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:57.365134 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:57.365207 (+ 73us) server_negotiation.cc:407] Connection header received
0929 19:10:57.365274 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:57.365275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:57.365289 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:57.365341 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:57.365666 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:57.366265 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:57.366992 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:57.367205 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:57.367280 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:57.369884 (+ 2604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.3:52829: BlockingRecv error: recv got EOF from 127.9.236.3:52829 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 29 19:10:58 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:58 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:58.105942 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:58.101359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.1:33209)
0929 19:10:58.101537 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:58.101547 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:58.101601 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:58.101757 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:58.101758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:58.101952 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:58.102020 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:58.102022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:58.102643 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:58.102644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:58.103257 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:58.103259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:58.103580 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:58.103580 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:58.103619 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:58.104109 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:58.104114 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:58.105865 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
W20260929 19:10:58.106045 19350 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:58.101448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:33209 (local address 127.9.236.62:39695)
0929 19:10:58.101588 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:58.101590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:58.101602 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:10:58.101666 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:58.101667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:58.101678 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:58.101751 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:58.102079 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:58.102583 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:58.103332 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:58.103521 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:58.103572 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:58.105967 (+ 2395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.1:33209: BlockingRecv error: recv got EOF from 127.9.236.1:33209 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
Sep 29 19:10:58 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:58 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:58.223767 19351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:58.218950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.2:38295)
0929 19:10:58.219097 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:58.219109 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:58.219161 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:58.219356 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:58.219356 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:58.219524 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:58.219577 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:58.219579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:58.220387 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:58.220387 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:58.220948 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:58.220950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:58.221354 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:58.221355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:58.221398 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:58.221901 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:58.221905 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:58.223703 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":63,"threads_started":1}
W20260929 19:10:58.223953 19350 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:58.219128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:38295 (local address 127.9.236.62:39695)
0929 19:10:58.219204 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:58.219206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:58.219214 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:10:58.219252 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:58.219253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:58.219267 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:58.219332 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:58.219684 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:58.220293 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:58.221033 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:58.221283 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:58.221344 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:58.223860 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.2:38295: BlockingRecv error: recv got EOF from 127.9.236.2:38295 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 29 19:10:58 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:58 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:58.375285 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:58.370399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.3:55719)
0929 19:10:58.370568 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:58.370579 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:58.370649 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:58.370789 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:58.370789 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:58.370997 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:58.371067 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:58.371069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:58.371682 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:58.371682 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:58.372300 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:58.372302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:58.372663 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:58.372664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:58.372701 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:58.373183 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:58.373187 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:58.375200 (+ 2013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":77,"threads_started":1}
W20260929 19:10:58.375377 19350 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:58.370446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:55719 (local address 127.9.236.62:39695)
0929 19:10:58.370496 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:58.370498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:58.370647 (+ 149us) server_negotiation.cc:407] Connection header received
0929 19:10:58.370712 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:58.370713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:58.370724 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:58.370769 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:58.371129 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:58.371612 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:58.372399 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:58.372589 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:58.372660 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:58.375315 (+ 2655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.3:55719: BlockingRecv error: recv got EOF from 127.9.236.3:55719 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 29 19:10:59 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:59 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:59.111634 19353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:59.106432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.1:44951)
0929 19:10:59.106608 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:59.106622 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:59.106703 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:59.106864 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:59.106865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:59.107094 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:59.107164 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:59.107165 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:59.107835 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:59.107836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:59.108471 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:59.108473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:59.108911 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:59.108912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:59.108975 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:59.109563 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:59.109568 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:59.111551 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":63,"threads_started":1}
W20260929 19:10:59.111688 19354 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:59.106562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:44951 (local address 127.9.236.62:39695)
0929 19:10:59.106700 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:59.106702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:59.106743 (+ 41us) server_negotiation.cc:407] Connection header received
0929 19:10:59.106767 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:59.106768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:59.106786 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:59.106844 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:59.107231 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:59.107739 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:59.108584 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:59.108814 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:59.108884 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:59.111627 (+ 2743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.1:44951: BlockingRecv error: recv got EOF from 127.9.236.1:44951 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
Sep 29 19:10:59 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:59 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:59.229811 19355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:59.224346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.2:56637)
0929 19:10:59.224501 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:59.224512 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:59.224598 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:59.224806 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:59.224807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:59.225005 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:59.225080 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:59.225082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:59.225884 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:59.225885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:59.226508 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:59.226510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:59.226943 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:59.226945 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:59.227047 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:59.227620 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:59.227624 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:59.229722 (+ 2098us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
W20260929 19:10:59.229928 19354 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:59.224460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:56637 (local address 127.9.236.62:39695)
0929 19:10:59.224537 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:59.224539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:59.224594 (+ 55us) server_negotiation.cc:407] Connection header received
0929 19:10:59.224655 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:59.224656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:59.224666 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:59.224774 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:59.225235 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:59.225754 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:59.226607 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:59.226841 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:59.226917 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:59.229864 (+ 2947us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.2:56637: BlockingRecv error: recv got EOF from 127.9.236.2:56637 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 29 19:10:59 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:10:59 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.3@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:10:59.381052 19356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:10:59.375809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.3:38631)
0929 19:10:59.375972 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 19:10:59.375983 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:10:59.376052 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:10:59.376176 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:10:59.376177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:10:59.376362 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:10:59.376439 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:59.376441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:59.377413 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:59.377415 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:59.378290 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:10:59.378294 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:10:59.378630 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:10:59.378631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:10:59.378672 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:59.379116 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:10:59.379120 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:10:59.380973 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":81,"threads_started":1}
W20260929 19:10:59.381186 19354 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:10:59.375872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:38631 (local address 127.9.236.62:39695)
0929 19:10:59.375943 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 19:10:59.375946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:10:59.376038 (+ 92us) server_negotiation.cc:407] Connection header received
0929 19:10:59.376104 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:10:59.376104 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:10:59.376118 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:10:59.376164 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:10:59.376513 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:59.377309 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:59.378367 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:10:59.378555 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:10:59.378611 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:10:59.381076 (+ 2465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.3:38631: BlockingRecv error: recv got EOF from 127.9.236.3:38631 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 29 19:11:00 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:11:00 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.1@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:11:00.118160 19357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:00.112284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.1:46161)
0929 19:11:00.112401 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:00.112413 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:00.112478 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:00.112716 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:00.112717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:00.112919 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:11:00.112993 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:00.112996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.114004 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.114006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:00.114874 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:00.114877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.115429 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.115430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:00.115485 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:00.115923 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:11:00.115927 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:11:00.118066 (+ 2139us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":65,"threads_started":1}
W20260929 19:11:00.118422 19358 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:11:00.112369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:46161 (local address 127.9.236.62:39695)
0929 19:11:00.112532 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:00.112535 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:00.112545 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:11:00.112578 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:00.112579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:00.112595 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:00.112826 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:00.113089 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.113922 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.115004 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.115362 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.115428 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:00.118197 (+ 2769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.1:46161: BlockingRecv error: recv got EOF from 127.9.236.1:46161 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":84,"threads_started":1}
Sep 29 19:11:00 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 19:11:00 dist-test-slave-wftq krb5kdc[18300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.236.2@KRBTEST.COM for oryx/127.9.236.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 19:11:00.235147 19359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:00.230471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39695 (local address 127.9.236.2:56231)
0929 19:11:00.230622 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:00.230635 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:00.230695 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:00.230897 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:00.230899 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:00.231125 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:11:00.231191 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:00.231193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.231838 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.231838 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:00.232420 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:00.232421 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.232755 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.232755 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:00.232796 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:00.233292 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:11:00.233298 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:11:00.235062 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.236.62:39695: Server oryx/127.9.236.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
W20260929 19:11:00.235227 19358 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:11:00.230614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:56231 (local address 127.9.236.62:39695)
0929 19:11:00.230699 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:00.230701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:00.230740 (+ 39us) server_negotiation.cc:407] Connection header received
0929 19:11:00.230772 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:00.230773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:00.230784 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:00.230833 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:00.231265 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.231768 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.232494 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.232690 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.232748 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:00.235169 (+ 2421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.236.2:56231: BlockingRecv error: recv got EOF from 127.9.236.2:56231 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 19:11:00.290087 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 18883
I20260929 19:11:00.294425 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 19035
I20260929 19:11:00.298422 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 19167
I20260929 19:11:00.302345 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 18813
2026-09-29T19:11:00Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17405 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-29T19:11:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:11:00Z Disabled control of system clock
I20260929 19:11:00.330224 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:37815
--webserver_interface=127.9.236.62
--webserver_port=0
--builtin_ntp_servers=127.9.236.20:43481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:37815
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 19:11:00.414968 19363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:11:00.415170 19363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:11:00.415190 19363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:11:00.416368 19363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:11:00.416412 19363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:11:00.416424 19363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:11:00.416435 19363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:11:00.416445 19363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:11:00.417810 19363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:43481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:37815
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:37815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:11:00.418042 19363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:11:00.418320 19363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:11:00.423480 19371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:00.423617 19363 server_base.cc:1034] running on GCE node
W20260929 19:11:00.423563 19369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:00.423475 19368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:00.424145 19363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:11:00.424479 19363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:11:00.425640 19363 hybrid_clock.cc:648] HybridClock initialized: now 1790709060425604 us; error 57 us; skew 500 ppm
I20260929 19:11:00.432299 19363 webserver.cc:460] Webserver started at http://127.9.236.62:39331/ using document root <none> and password file <none>
I20260929 19:11:00.432567 19363 fs_manager.cc:359] Metadata directory not provided
I20260929 19:11:00.432622 19363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:11:00.432765 19363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:11:00.433997 19363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "5166b1f8a2074cc6891496ff09c27681"
format_stamp: "Formatted at 2026-09-29 19:11:00 on dist-test-slave-wftq"
server_key: "414b7aaad609c701c6bc3e3f946faacd"
server_key_iv: "b8e713534e2b27fb072bd5a843c66c64"
server_key_version: "encryptionkey@0"
I20260929 19:11:00.434345 19363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "5166b1f8a2074cc6891496ff09c27681"
format_stamp: "Formatted at 2026-09-29 19:11:00 on dist-test-slave-wftq"
server_key: "414b7aaad609c701c6bc3e3f946faacd"
server_key_iv: "b8e713534e2b27fb072bd5a843c66c64"
server_key_version: "encryptionkey@0"
I20260929 19:11:00.435667 19363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:11:00.436467 19377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:00.436723 19363 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:11:00.436815 19363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "5166b1f8a2074cc6891496ff09c27681"
format_stamp: "Formatted at 2026-09-29 19:11:00 on dist-test-slave-wftq"
server_key: "414b7aaad609c701c6bc3e3f946faacd"
server_key_iv: "b8e713534e2b27fb072bd5a843c66c64"
server_key_version: "encryptionkey@0"
I20260929 19:11:00.436877 19363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:11:00.448668 19363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:11:00.448997 19363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:11:00.449182 19363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:11:00.453548 19363 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:37815
I20260929 19:11:00.453596 19429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:37815 every 8 connection(s)
I20260929 19:11:00.453940 19363 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:11:00.454660 19430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:11:00.455109 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 19363
I20260929 19:11:00.455232 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:11:00.455430 10160 external_mini_cluster.cc:1428] Setting key 6b615080fc23ed2bec961415be4580e7
I20260929 19:11:00.457867 19430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681: Bootstrap starting.
I20260929 19:11:00.458439 19430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681: Neither blocks nor log segments found. Creating new log.
I20260929 19:11:00.458742 19430 log.cc:824] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681: Log is configured to *not* fsync() on all Append() calls
I20260929 19:11:00.459513 19430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681: No bootstrap required, opened a new log
I20260929 19:11:00.461257 19433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:00.456056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37100 (local address 127.9.236.62:37815)
0929 19:11:00.456302 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:00.456316 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:00.456347 (+ 31us) server_negotiation.cc:407] Connection header received
0929 19:11:00.456907 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:00.456914 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:00.456990 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:00.457176 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:00.457388 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.458357 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.459313 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.459763 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.459870 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:00.460534 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:11:00.460538 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:11:00.460538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:11:00.460724 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:11:00.460760 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:11:00.460771 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:11:00.460822 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:00.460924 (+ 102us) server_negotiation.cc:299] Negotiation successful
0929 19:11:00.460998 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":104,"threads_started":1}
I20260929 19:11:00.461792 19430 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5166b1f8a2074cc6891496ff09c27681" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37815 } }
I20260929 19:11:00.461941 19430 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:11:00.461973 19430 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5166b1f8a2074cc6891496ff09c27681, State: Initialized, Role: FOLLOWER
I20260929 19:11:00.462086 19430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5166b1f8a2074cc6891496ff09c27681" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37815 } }
I20260929 19:11:00.462168 19430 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:11:00.462234 19430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:11:00.462283 19430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:11:00.462810 19430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5166b1f8a2074cc6891496ff09c27681" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37815 } }
I20260929 19:11:00.462898 19430 leader_election.cc:302] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5166b1f8a2074cc6891496ff09c27681; no voters:
I20260929 19:11:00.463071 19430 leader_election.cc:288] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:11:00.463150 19435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:11:00.463315 19430 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:11:00.463344 19435 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681 [term 1 LEADER]: Becoming Leader. State: Replica: 5166b1f8a2074cc6891496ff09c27681, State: Running, Role: LEADER
I20260929 19:11:00.463445 19435 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5166b1f8a2074cc6891496ff09c27681" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37815 } }
I20260929 19:11:00.463830 19436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5166b1f8a2074cc6891496ff09c27681" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5166b1f8a2074cc6891496ff09c27681" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37815 } } }
I20260929 19:11:00.463840 19437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5166b1f8a2074cc6891496ff09c27681. Latest consensus state: current_term: 1 leader_uuid: "5166b1f8a2074cc6891496ff09c27681" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5166b1f8a2074cc6891496ff09c27681" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37815 } } }
I20260929 19:11:00.464010 19436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681 [sys.catalog]: This master's current role is: LEADER
I20260929 19:11:00.464017 19437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681 [sys.catalog]: This master's current role is: LEADER
I20260929 19:11:00.464315 19444 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:11:00.464905 19444 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:11:00.470248 19444 catalog_manager.cc:1347] Generated new cluster ID: 6fe155fe66944ffe937aec9bdedcc2a2
I20260929 19:11:00.470306 19444 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:11:00.487259 19444 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:11:00.487804 19444 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:11:00.490398 19444 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5166b1f8a2074cc6891496ff09c27681: Generated new TSK 0
I20260929 19:11:00.490561 19444 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:11:00.491837 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:37815
--builtin_ntp_servers=127.9.236.20:43481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260929 19:11:00.577299 19454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:11:00.577534 19454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:11:00.577554 19454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:11:00.578804 19454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:11:00.578848 19454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:11:00.578872 19454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:11:00.580209 19454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:43481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:37815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:11:00.580449 19454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:11:00.580760 19454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:11:00.585904 19462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:00.585949 19459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:00.586009 19460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:00.586387 19454 server_base.cc:1034] running on GCE node
I20260929 19:11:00.586624 19454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:11:00.586871 19454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:11:00.588021 19454 hybrid_clock.cc:648] HybridClock initialized: now 1790709060588005 us; error 40 us; skew 500 ppm
I20260929 19:11:00.589403 19454 webserver.cc:460] Webserver started at http://127.9.236.1:40937/ using document root <none> and password file <none>
I20260929 19:11:00.589648 19454 fs_manager.cc:359] Metadata directory not provided
I20260929 19:11:00.589696 19454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:11:00.589831 19454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:11:00.590965 19454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "c9ddd0648fc645feb0f95d37e5a091f0"
format_stamp: "Formatted at 2026-09-29 19:11:00 on dist-test-slave-wftq"
server_key: "10122f2df1bd2931d8bfd68db32b381e"
server_key_iv: "43f0d415854ae5737ff0fb3a4624003d"
server_key_version: "encryptionkey@0"
I20260929 19:11:00.591325 19454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "c9ddd0648fc645feb0f95d37e5a091f0"
format_stamp: "Formatted at 2026-09-29 19:11:00 on dist-test-slave-wftq"
server_key: "10122f2df1bd2931d8bfd68db32b381e"
server_key_iv: "43f0d415854ae5737ff0fb3a4624003d"
server_key_version: "encryptionkey@0"
I20260929 19:11:00.592667 19454 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:11:00.593482 19468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:00.593691 19454 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:11:00.593776 19454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "c9ddd0648fc645feb0f95d37e5a091f0"
format_stamp: "Formatted at 2026-09-29 19:11:00 on dist-test-slave-wftq"
server_key: "10122f2df1bd2931d8bfd68db32b381e"
server_key_iv: "43f0d415854ae5737ff0fb3a4624003d"
server_key_version: "encryptionkey@0"
I20260929 19:11:00.593843 19454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:11:00.608848 19454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:11:00.609223 19454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:11:00.609362 19454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:11:00.609592 19454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:11:00.609956 19454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:11:00.609989 19454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:00.610020 19454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:11:00.610037 19454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:00.616704 19454 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:36927
I20260929 19:11:00.616791 19581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:36927 every 8 connection(s)
I20260929 19:11:00.617192 19454 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:11:00.622453 19433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:00.618007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:46105 (local address 127.9.236.62:37815)
0929 19:11:00.618069 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:00.618072 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:00.618321 (+ 249us) server_negotiation.cc:407] Connection header received
0929 19:11:00.618833 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:00.618834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:00.618844 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:00.618923 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:00.619657 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.620200 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.621205 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.621402 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.621454 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:00.622211 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:11:00.622212 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:11:00.622213 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:11:00.622217 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:11:00.622233 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:11:00.622238 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:11:00.622278 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:00.622356 (+ 78us) server_negotiation.cc:299] Negotiation successful
0929 19:11:00.622387 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 19:11:00.622624 19584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:00.617964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37815 (local address 127.9.236.1:46105)
0929 19:11:00.618176 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:00.618217 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:00.618737 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:00.619007 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:00.619011 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:00.619019 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:11:00.619554 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:00.619558 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.620301 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.620302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:00.621132 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:00.621134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.621465 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.621466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:00.621572 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:00.622084 (+ 512us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:11:00.622085 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:11:00.622112 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:11:00.622135 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:11:00.622289 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:11:00.622292 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:11:00.622294 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:11:00.622346 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 19:11:00.622423 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":97,"threads_started":1}
I20260929 19:11:00.623196 19582 heartbeater.cc:347] Connected to a master server at 127.9.236.62:37815
I20260929 19:11:00.623330 19582 heartbeater.cc:464] Registering TS with master...
I20260929 19:11:00.623512 19582 heartbeater.cc:511] Master 127.9.236.62:37815 requested a full tablet report, sending...
I20260929 19:11:00.624069 19394 ts_manager.cc:194] Registered new tserver with Master: c9ddd0648fc645feb0f95d37e5a091f0 (127.9.236.1:36927)
I20260929 19:11:00.624744 19394 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.236.1:46105
I20260929 19:11:00.627182 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 19454
I20260929 19:11:00.627270 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:11:00.627475 10160 external_mini_cluster.cc:1428] Setting key 3a380507db97031bf295fca799011234
I20260929 19:11:00.628391 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:37815
--builtin_ntp_servers=127.9.236.20:43481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260929 19:11:00.711831 19585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:11:00.712042 19585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:11:00.712062 19585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:11:00.713245 19585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:11:00.713294 19585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:11:00.713318 19585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:11:00.714637 19585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:43481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:37815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:11:00.714871 19585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:11:00.715153 19585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:11:00.720474 19591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:00.720505 19593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:00.720511 19585 server_base.cc:1034] running on GCE node
W20260929 19:11:00.720453 19590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:00.720927 19585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:11:00.721172 19585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:11:00.722360 19585 hybrid_clock.cc:648] HybridClock initialized: now 1790709060722338 us; error 44 us; skew 500 ppm
I20260929 19:11:00.723752 19585 webserver.cc:460] Webserver started at http://127.9.236.2:41473/ using document root <none> and password file <none>
I20260929 19:11:00.723971 19585 fs_manager.cc:359] Metadata directory not provided
I20260929 19:11:00.724020 19585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:11:00.724135 19585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:11:00.725219 19585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "6ef27494a25541339a46dab16dedf198"
format_stamp: "Formatted at 2026-09-29 19:11:00 on dist-test-slave-wftq"
server_key: "2c0a94abe17610b006021e98eab3e522"
server_key_iv: "049d7ca5d84d5ec70a10fc2698cca3c0"
server_key_version: "encryptionkey@0"
I20260929 19:11:00.725538 19585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "6ef27494a25541339a46dab16dedf198"
format_stamp: "Formatted at 2026-09-29 19:11:00 on dist-test-slave-wftq"
server_key: "2c0a94abe17610b006021e98eab3e522"
server_key_iv: "049d7ca5d84d5ec70a10fc2698cca3c0"
server_key_version: "encryptionkey@0"
I20260929 19:11:00.726796 19585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:11:00.727569 19599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:00.727768 19585 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:11:00.727862 19585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "6ef27494a25541339a46dab16dedf198"
format_stamp: "Formatted at 2026-09-29 19:11:00 on dist-test-slave-wftq"
server_key: "2c0a94abe17610b006021e98eab3e522"
server_key_iv: "049d7ca5d84d5ec70a10fc2698cca3c0"
server_key_version: "encryptionkey@0"
I20260929 19:11:00.727921 19585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:11:00.738713 19585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:11:00.738972 19585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:11:00.739091 19585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:11:00.739274 19585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:11:00.739604 19585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:11:00.739645 19585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:00.739665 19585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:11:00.739703 19585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:00.746001 19585 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:39591
I20260929 19:11:00.746091 19712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:39591 every 8 connection(s)
I20260929 19:11:00.746376 19585 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:11:00.752013 19433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:00.747086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:53793 (local address 127.9.236.62:37815)
0929 19:11:00.747173 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:00.747176 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:00.747395 (+ 219us) server_negotiation.cc:407] Connection header received
0929 19:11:00.748024 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:00.748026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:00.748037 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:00.748129 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:00.748897 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.749472 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.750608 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.750812 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.750876 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:00.751765 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:11:00.751766 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:11:00.751767 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:11:00.751771 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:11:00.751787 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:11:00.751791 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:11:00.751842 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:00.751917 (+ 75us) server_negotiation.cc:299] Negotiation successful
0929 19:11:00.751947 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 19:11:00.752238 19715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:00.747043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37815 (local address 127.9.236.2:53793)
0929 19:11:00.747246 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:00.747289 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:00.747888 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:00.748199 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:00.748203 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:00.748212 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:11:00.748797 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:00.748801 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.749595 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.749596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:00.750511 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:00.750514 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.750889 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.750889 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:00.751003 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:00.751564 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:11:00.751565 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:11:00.751597 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:11:00.751634 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:11:00.751851 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:11:00.751855 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:11:00.751857 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:11:00.751909 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 19:11:00.752010 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260929 19:11:00.752763 19713 heartbeater.cc:347] Connected to a master server at 127.9.236.62:37815
I20260929 19:11:00.752861 19713 heartbeater.cc:464] Registering TS with master...
I20260929 19:11:00.753062 19713 heartbeater.cc:511] Master 127.9.236.62:37815 requested a full tablet report, sending...
I20260929 19:11:00.753525 19394 ts_manager.cc:194] Registered new tserver with Master: 6ef27494a25541339a46dab16dedf198 (127.9.236.2:39591)
I20260929 19:11:00.753593 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 19585
I20260929 19:11:00.753690 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:11:00.753886 10160 external_mini_cluster.cc:1428] Setting key 0620be81cb5c3a9a2c2834b2c099cf08
I20260929 19:11:00.754051 19394 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.236.2:53793
I20260929 19:11:00.754987 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:37815
--builtin_ntp_servers=127.9.236.20:43481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260929 19:11:00.833667 19716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:11:00.833866 19716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:11:00.833886 19716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:11:00.834997 19716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:11:00.835038 19716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:11:00.835062 19716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:11:00.836352 19716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:43481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:37815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:11:00.836562 19716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:11:00.836827 19716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:11:00.842123 19722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:00.842139 19724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:00.842134 19721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:00.842180 19716 server_base.cc:1034] running on GCE node
I20260929 19:11:00.842564 19716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:11:00.842799 19716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:11:00.843973 19716 hybrid_clock.cc:648] HybridClock initialized: now 1790709060843939 us; error 47 us; skew 500 ppm
I20260929 19:11:00.845238 19716 webserver.cc:460] Webserver started at http://127.9.236.3:44055/ using document root <none> and password file <none>
I20260929 19:11:00.845445 19716 fs_manager.cc:359] Metadata directory not provided
I20260929 19:11:00.845492 19716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:11:00.845615 19716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:11:00.846652 19716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "dc3dc90f0c944d59877fe2534b4f4091"
format_stamp: "Formatted at 2026-09-29 19:11:00 on dist-test-slave-wftq"
server_key: "bab0159f3963c214c6f6fb5df84e5ae5"
server_key_iv: "c557757d8bd30d8a3b2b850cec8b4318"
server_key_version: "encryptionkey@0"
I20260929 19:11:00.846976 19716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "dc3dc90f0c944d59877fe2534b4f4091"
format_stamp: "Formatted at 2026-09-29 19:11:00 on dist-test-slave-wftq"
server_key: "bab0159f3963c214c6f6fb5df84e5ae5"
server_key_iv: "c557757d8bd30d8a3b2b850cec8b4318"
server_key_version: "encryptionkey@0"
I20260929 19:11:00.848213 19716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:11:00.848989 19730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:00.849206 19716 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:11:00.849289 19716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "dc3dc90f0c944d59877fe2534b4f4091"
format_stamp: "Formatted at 2026-09-29 19:11:00 on dist-test-slave-wftq"
server_key: "bab0159f3963c214c6f6fb5df84e5ae5"
server_key_iv: "c557757d8bd30d8a3b2b850cec8b4318"
server_key_version: "encryptionkey@0"
I20260929 19:11:00.849354 19716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:11:00.867139 19716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:11:00.867417 19716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:11:00.867543 19716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:11:00.867789 19716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:11:00.868100 19716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:11:00.868134 19716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:00.868162 19716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:11:00.868178 19716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:00.874598 19716 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:34847
I20260929 19:11:00.874658 19843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:34847 every 8 connection(s)
I20260929 19:11:00.874955 19716 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:11:00.879984 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 19716
I20260929 19:11:00.880103 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:11:00.880299 10160 external_mini_cluster.cc:1428] Setting key 909a3fb51349e83eecdcd177d26470cf
I20260929 19:11:00.880543 19433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:00.875763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:56609 (local address 127.9.236.62:37815)
0929 19:11:00.875833 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:00.875835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:00.876095 (+ 260us) server_negotiation.cc:407] Connection header received
0929 19:11:00.876724 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:00.876725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:00.876737 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:00.876802 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:00.877540 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.878049 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.879119 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.879319 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.879389 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:00.880255 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:11:00.880257 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:11:00.880257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:11:00.880261 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:11:00.880275 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:11:00.880279 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:11:00.880333 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:00.880440 (+ 107us) server_negotiation.cc:299] Negotiation successful
0929 19:11:00.880467 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 19:11:00.880757 19846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:00.875689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37815 (local address 127.9.236.3:56609)
0929 19:11:00.875931 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:00.875983 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:00.876577 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:00.876868 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:00.876871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:00.876880 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:11:00.877460 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:00.877465 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.878124 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.878125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:00.879022 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:00.879025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:00.879398 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:00.879398 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:00.879510 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:00.880088 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:11:00.880089 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:11:00.880131 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:11:00.880162 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:11:00.880352 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:11:00.880356 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:11:00.880359 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:11:00.880431 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 19:11:00.880537 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":84,"threads_started":1}
I20260929 19:11:00.881258 19844 heartbeater.cc:347] Connected to a master server at 127.9.236.62:37815
I20260929 19:11:00.881345 19844 heartbeater.cc:464] Registering TS with master...
I20260929 19:11:00.881522 19844 heartbeater.cc:511] Master 127.9.236.62:37815 requested a full tablet report, sending...
I20260929 19:11:00.881897 19394 ts_manager.cc:194] Registered new tserver with Master: dc3dc90f0c944d59877fe2534b4f4091 (127.9.236.3:34847)
I20260929 19:11:00.882303 19394 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.236.3:56609
I20260929 19:11:00.892385 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 19:11:00.894559 19433 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:11:00.894086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37112 (local address 127.9.236.62:37815)
0929 19:11:00.894173 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:00.894175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:00.894243 (+ 68us) server_negotiation.cc:407] Connection header received
0929 19:11:00.894315 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:00.894316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:00.894325 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:00.894372 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:00.894509 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37112: BlockingRecv error: recv got EOF from 127.0.0.1:37112 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 19:11:00.894833 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 19454
I20260929 19:11:00.899281 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 19585
I20260929 19:11:00.903278 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 19716
I20260929 19:11:00.907823 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 19363
2026-09-29T19:11:00Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (604 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-29T19:11:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:11:00Z Disabled control of system clock
I20260929 19:11:00.934273 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:43477
--webserver_interface=127.9.236.62
--webserver_port=0
--builtin_ntp_servers=127.9.236.20:44471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:43477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 19:11:01.011132 19855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:11:01.011344 19855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:11:01.011363 19855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:11:01.012459 19855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:11:01.012498 19855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:11:01.012511 19855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:11:01.012521 19855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:11:01.012532 19855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:11:01.013890 19855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:44471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:43477
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:43477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:11:01.014084 19855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:11:01.014314 19855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:11:01.019330 19855 server_base.cc:1034] running on GCE node
W20260929 19:11:01.019281 19861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:01.019459 19863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:01.019284 19860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:01.019847 19855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:11:01.020123 19855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:11:01.021263 19855 hybrid_clock.cc:648] HybridClock initialized: now 1790709061021251 us; error 35 us; skew 500 ppm
I20260929 19:11:01.022476 19855 webserver.cc:460] Webserver started at http://127.9.236.62:45991/ using document root <none> and password file <none>
I20260929 19:11:01.022701 19855 fs_manager.cc:359] Metadata directory not provided
I20260929 19:11:01.022749 19855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:11:01.022872 19855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:11:01.023895 19855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "44f3b5a9a87b4493aac52fed54e48bfc"
format_stamp: "Formatted at 2026-09-29 19:11:01 on dist-test-slave-wftq"
server_key: "ddc33f2d592a6cec03a68321bfc76a60"
server_key_iv: "b57078bc699cbeb6032c0f29f42195fc"
server_key_version: "encryptionkey@0"
I20260929 19:11:01.024202 19855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "44f3b5a9a87b4493aac52fed54e48bfc"
format_stamp: "Formatted at 2026-09-29 19:11:01 on dist-test-slave-wftq"
server_key: "ddc33f2d592a6cec03a68321bfc76a60"
server_key_iv: "b57078bc699cbeb6032c0f29f42195fc"
server_key_version: "encryptionkey@0"
I20260929 19:11:01.025490 19855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:11:01.026327 19869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:01.026510 19855 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:11:01.026592 19855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "44f3b5a9a87b4493aac52fed54e48bfc"
format_stamp: "Formatted at 2026-09-29 19:11:01 on dist-test-slave-wftq"
server_key: "ddc33f2d592a6cec03a68321bfc76a60"
server_key_iv: "b57078bc699cbeb6032c0f29f42195fc"
server_key_version: "encryptionkey@0"
I20260929 19:11:01.026650 19855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:11:01.029932 19855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:11:01.030165 19855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:11:01.030277 19855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:11:01.034245 19855 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:43477
I20260929 19:11:01.034353 19921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:43477 every 8 connection(s)
I20260929 19:11:01.034602 19855 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:11:01.035346 19922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:11:01.038030 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 19855
I20260929 19:11:01.038115 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:11:01.038321 10160 external_mini_cluster.cc:1428] Setting key f7e91507730046c6298ca90b95ed404a
I20260929 19:11:01.038728 19922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc: Bootstrap starting.
I20260929 19:11:01.039485 19922 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc: Neither blocks nor log segments found. Creating new log.
I20260929 19:11:01.039803 19922 log.cc:824] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc: Log is configured to *not* fsync() on all Append() calls
I20260929 19:11:01.040683 19922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc: No bootstrap required, opened a new log
I20260929 19:11:01.041282 19925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:01.038949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53662 (local address 127.9.236.62:43477)
0929 19:11:01.039143 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:01.039150 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:01.039184 (+ 34us) server_negotiation.cc:407] Connection header received
0929 19:11:01.039789 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:01.039796 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:01.039838 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:01.039989 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:01.040802 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:11:01.040805 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:11:01.040806 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:11:01.040810 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:11:01.040852 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:11:01.040862 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:11:01.040938 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:01.040987 (+ 49us) server_negotiation.cc:299] Negotiation successful
0929 19:11:01.041043 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
I20260929 19:11:01.042992 19922 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f3b5a9a87b4493aac52fed54e48bfc" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43477 } }
I20260929 19:11:01.043183 19922 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:11:01.043244 19922 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44f3b5a9a87b4493aac52fed54e48bfc, State: Initialized, Role: FOLLOWER
I20260929 19:11:01.043407 19922 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f3b5a9a87b4493aac52fed54e48bfc" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43477 } }
I20260929 19:11:01.043503 19922 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:11:01.043546 19922 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:11:01.043596 19922 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:11:01.044184 19922 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f3b5a9a87b4493aac52fed54e48bfc" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43477 } }
I20260929 19:11:01.044286 19922 leader_election.cc:302] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44f3b5a9a87b4493aac52fed54e48bfc; no voters:
I20260929 19:11:01.044437 19922 leader_election.cc:288] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:11:01.044567 19927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:11:01.044772 19927 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc [term 1 LEADER]: Becoming Leader. State: Replica: 44f3b5a9a87b4493aac52fed54e48bfc, State: Running, Role: LEADER
I20260929 19:11:01.044754 19922 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:11:01.044958 19927 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f3b5a9a87b4493aac52fed54e48bfc" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43477 } }
I20260929 19:11:01.045435 19929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44f3b5a9a87b4493aac52fed54e48bfc. Latest consensus state: current_term: 1 leader_uuid: "44f3b5a9a87b4493aac52fed54e48bfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f3b5a9a87b4493aac52fed54e48bfc" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43477 } } }
I20260929 19:11:01.045439 19928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44f3b5a9a87b4493aac52fed54e48bfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f3b5a9a87b4493aac52fed54e48bfc" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43477 } } }
I20260929 19:11:01.045605 19929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc [sys.catalog]: This master's current role is: LEADER
I20260929 19:11:01.045634 19928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc [sys.catalog]: This master's current role is: LEADER
I20260929 19:11:01.046334 19936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:11:01.046875 19936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:11:01.048499 19936 catalog_manager.cc:1347] Generated new cluster ID: 4e15c81fd3ce4b7886be4d35e9c4c805
I20260929 19:11:01.048555 19936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:11:01.073230 19936 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:11:01.073428 19936 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:11:01.081987 19936 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44f3b5a9a87b4493aac52fed54e48bfc: Generated new TSK 0
I20260929 19:11:01.082214 19936 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:11:01.085552 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:43477
--builtin_ntp_servers=127.9.236.20:44471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 19:11:01.171976 19946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:11:01.172216 19946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:11:01.172240 19946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:11:01.173508 19946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:11:01.173549 19946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:11:01.173574 19946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:11:01.174957 19946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:44471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:43477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:11:01.175232 19946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:11:01.175559 19946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:11:01.180970 19952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:01.180999 19954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:01.180984 19951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:01.181272 19946 server_base.cc:1034] running on GCE node
I20260929 19:11:01.181476 19946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:11:01.181720 19946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:11:01.182864 19946 hybrid_clock.cc:648] HybridClock initialized: now 1790709061182831 us; error 45 us; skew 500 ppm
I20260929 19:11:01.184209 19946 webserver.cc:460] Webserver started at http://127.9.236.1:36181/ using document root <none> and password file <none>
I20260929 19:11:01.184451 19946 fs_manager.cc:359] Metadata directory not provided
I20260929 19:11:01.184499 19946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:11:01.184708 19946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:11:01.185899 19946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "7041f42e8d7c4d6198aaf6af47c68cdc"
format_stamp: "Formatted at 2026-09-29 19:11:01 on dist-test-slave-wftq"
server_key: "7d433160e448e20b4d7bfb8b83864a83"
server_key_iv: "068230fe94e1d179e952301f6c64ca1f"
server_key_version: "encryptionkey@0"
I20260929 19:11:01.186255 19946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "7041f42e8d7c4d6198aaf6af47c68cdc"
format_stamp: "Formatted at 2026-09-29 19:11:01 on dist-test-slave-wftq"
server_key: "7d433160e448e20b4d7bfb8b83864a83"
server_key_iv: "068230fe94e1d179e952301f6c64ca1f"
server_key_version: "encryptionkey@0"
I20260929 19:11:01.187598 19946 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:11:01.188453 19960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:01.188688 19946 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:11:01.188776 19946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "7041f42e8d7c4d6198aaf6af47c68cdc"
format_stamp: "Formatted at 2026-09-29 19:11:01 on dist-test-slave-wftq"
server_key: "7d433160e448e20b4d7bfb8b83864a83"
server_key_iv: "068230fe94e1d179e952301f6c64ca1f"
server_key_version: "encryptionkey@0"
I20260929 19:11:01.188845 19946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:11:01.192350 19946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:11:01.192631 19946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:11:01.192763 19946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:11:01.192970 19946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:11:01.193317 19946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:11:01.193351 19946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:01.193378 19946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:11:01.193403 19946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:01.200100 19946 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:36405
I20260929 19:11:01.200163 20073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:36405 every 8 connection(s)
I20260929 19:11:01.200498 19946 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:11:01.200837 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 19946
I20260929 19:11:01.200951 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:11:01.201187 10160 external_mini_cluster.cc:1428] Setting key 57691b4ace62c8216751d1a1a9ac60a9
I20260929 19:11:01.202312 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:43477
--builtin_ntp_servers=127.9.236.20:44471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260929 19:11:01.204037 19925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:01.201350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:33889 (local address 127.9.236.62:43477)
0929 19:11:01.201406 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:01.201409 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:01.201747 (+ 338us) server_negotiation.cc:407] Connection header received
0929 19:11:01.202319 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:01.202320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:01.202327 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:01.202370 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:01.203773 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:11:01.203774 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:11:01.203774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:11:01.203779 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:11:01.203801 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:11:01.203805 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:11:01.203871 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:01.203939 (+ 68us) server_negotiation.cc:299] Negotiation successful
0929 19:11:01.203955 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:11:01.204167 20076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:01.201352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:43477 (local address 127.9.236.1:33889)
0929 19:11:01.201571 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:01.201615 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:01.202204 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:01.202470 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:01.202474 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:01.202483 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:11:01.203569 (+ 1086us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:11:01.203570 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:11:01.203617 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:11:01.203658 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:11:01.203870 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:11:01.203873 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:11:01.203875 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:11:01.203934 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 19:11:01.203984 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":116,"threads_started":1}
I20260929 19:11:01.204742 20074 heartbeater.cc:347] Connected to a master server at 127.9.236.62:43477
I20260929 19:11:01.204839 20074 heartbeater.cc:464] Registering TS with master...
I20260929 19:11:01.204977 20074 heartbeater.cc:511] Master 127.9.236.62:43477 requested a full tablet report, sending...
I20260929 19:11:01.205614 19886 ts_manager.cc:194] Registered new tserver with Master: 7041f42e8d7c4d6198aaf6af47c68cdc (127.9.236.1:36405)
W20260929 19:11:01.285526 20077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:11:01.285753 20077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:11:01.285782 20077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:11:01.286936 20077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:11:01.286973 20077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:11:01.287006 20077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:11:01.288304 20077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:44471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:43477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:11:01.288524 20077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:11:01.288828 20077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:11:01.294148 20083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:01.294278 20085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:01.294334 20077 server_base.cc:1034] running on GCE node
W20260929 19:11:01.294153 20082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:01.294663 20077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:11:01.294865 20077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:11:01.295993 20077 hybrid_clock.cc:648] HybridClock initialized: now 1790709061296008 us; error 53 us; skew 500 ppm
I20260929 19:11:01.297240 20077 webserver.cc:460] Webserver started at http://127.9.236.2:36499/ using document root <none> and password file <none>
I20260929 19:11:01.297451 20077 fs_manager.cc:359] Metadata directory not provided
I20260929 19:11:01.297498 20077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:11:01.297623 20077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:11:01.298650 20077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "5cd591522d224944a94ac0dec11f8b47"
format_stamp: "Formatted at 2026-09-29 19:11:01 on dist-test-slave-wftq"
server_key: "1da53b69eda2cb2d843e7544c319337b"
server_key_iv: "077c3b97437b210e8d890f1535a8c568"
server_key_version: "encryptionkey@0"
I20260929 19:11:01.298975 20077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "5cd591522d224944a94ac0dec11f8b47"
format_stamp: "Formatted at 2026-09-29 19:11:01 on dist-test-slave-wftq"
server_key: "1da53b69eda2cb2d843e7544c319337b"
server_key_iv: "077c3b97437b210e8d890f1535a8c568"
server_key_version: "encryptionkey@0"
I20260929 19:11:01.300299 20077 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:11:01.301035 20091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:01.301237 20077 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:11:01.301331 20077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "5cd591522d224944a94ac0dec11f8b47"
format_stamp: "Formatted at 2026-09-29 19:11:01 on dist-test-slave-wftq"
server_key: "1da53b69eda2cb2d843e7544c319337b"
server_key_iv: "077c3b97437b210e8d890f1535a8c568"
server_key_version: "encryptionkey@0"
I20260929 19:11:01.301400 20077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:11:01.304567 20077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:11:01.304801 20077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:11:01.304914 20077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:11:01.305132 20077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:11:01.305423 20077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:11:01.305455 20077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:01.305480 20077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:11:01.305495 20077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:01.311522 20077 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:33743
I20260929 19:11:01.311594 20204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:33743 every 8 connection(s)
I20260929 19:11:01.311866 20077 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:11:01.315129 19925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:01.312624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:57901 (local address 127.9.236.62:43477)
0929 19:11:01.312718 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:01.312721 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:01.312956 (+ 235us) server_negotiation.cc:407] Connection header received
0929 19:11:01.313552 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:01.313553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:01.313565 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:01.313636 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:01.314861 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:11:01.314862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:11:01.314862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:11:01.314866 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:11:01.314880 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:11:01.314885 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:11:01.314950 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:01.315058 (+ 108us) server_negotiation.cc:299] Negotiation successful
0929 19:11:01.315073 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 19:11:01.315291 20207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:01.312570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:43477 (local address 127.9.236.2:57901)
0929 19:11:01.312776 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:01.312816 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:01.313453 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:01.313700 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:01.313704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:01.313713 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:11:01.314701 (+ 988us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:11:01.314701 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:11:01.314735 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:11:01.314767 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:11:01.314954 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:11:01.314957 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:11:01.314959 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:11:01.315024 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 19:11:01.315079 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
I20260929 19:11:01.315846 20205 heartbeater.cc:347] Connected to a master server at 127.9.236.62:43477
I20260929 19:11:01.315936 20205 heartbeater.cc:464] Registering TS with master...
I20260929 19:11:01.316098 20205 heartbeater.cc:511] Master 127.9.236.62:43477 requested a full tablet report, sending...
I20260929 19:11:01.316504 19886 ts_manager.cc:194] Registered new tserver with Master: 5cd591522d224944a94ac0dec11f8b47 (127.9.236.2:33743)
I20260929 19:11:01.318282 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 20077
I20260929 19:11:01.318372 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:11:01.318535 10160 external_mini_cluster.cc:1428] Setting key 378f1143c788e107ae145f6ee9331951
I20260929 19:11:01.319470 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:43477
--builtin_ntp_servers=127.9.236.20:44471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 19:11:01.410069 20208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:11:01.410285 20208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:11:01.410308 20208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:11:01.411500 20208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:11:01.411542 20208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:11:01.411567 20208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:11:01.412846 20208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:44471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:43477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:11:01.413039 20208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:11:01.413336 20208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:11:01.418571 20213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:01.418593 20208 server_base.cc:1034] running on GCE node
W20260929 19:11:01.418546 20214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:01.418593 20216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:01.418969 20208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:11:01.419214 20208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:11:01.420382 20208 hybrid_clock.cc:648] HybridClock initialized: now 1790709061420361 us; error 47 us; skew 500 ppm
I20260929 19:11:01.421707 20208 webserver.cc:460] Webserver started at http://127.9.236.3:36261/ using document root <none> and password file <none>
I20260929 19:11:01.421936 20208 fs_manager.cc:359] Metadata directory not provided
I20260929 19:11:01.421986 20208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:11:01.422115 20208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:11:01.423177 20208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "683e8b2b172a4515b3530c69353b626a"
format_stamp: "Formatted at 2026-09-29 19:11:01 on dist-test-slave-wftq"
server_key: "955a0700385d05a59d854b01fa2e0cc9"
server_key_iv: "7f39d108074618269f5251a90e39cb27"
server_key_version: "encryptionkey@0"
I20260929 19:11:01.423499 20208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "683e8b2b172a4515b3530c69353b626a"
format_stamp: "Formatted at 2026-09-29 19:11:01 on dist-test-slave-wftq"
server_key: "955a0700385d05a59d854b01fa2e0cc9"
server_key_iv: "7f39d108074618269f5251a90e39cb27"
server_key_version: "encryptionkey@0"
I20260929 19:11:01.424767 20208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:11:01.425550 20222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:01.425742 20208 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 19:11:01.425832 20208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "683e8b2b172a4515b3530c69353b626a"
format_stamp: "Formatted at 2026-09-29 19:11:01 on dist-test-slave-wftq"
server_key: "955a0700385d05a59d854b01fa2e0cc9"
server_key_iv: "7f39d108074618269f5251a90e39cb27"
server_key_version: "encryptionkey@0"
I20260929 19:11:01.425894 20208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:11:01.429157 20208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:11:01.429394 20208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:11:01.429505 20208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:11:01.429678 20208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:11:01.429975 20208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:11:01.430007 20208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:01.430032 20208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:11:01.430047 20208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:01.436103 20208 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:44993
I20260929 19:11:01.436159 20335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:44993 every 8 connection(s)
I20260929 19:11:01.436442 20208 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:11:01.439447 19925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:01.437254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:60081 (local address 127.9.236.62:43477)
0929 19:11:01.437350 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:01.437354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:01.437569 (+ 215us) server_negotiation.cc:407] Connection header received
0929 19:11:01.438032 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:01.438034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:01.438041 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:01.438102 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:01.439189 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:11:01.439190 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:11:01.439191 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:11:01.439195 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:11:01.439217 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:11:01.439223 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:11:01.439295 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:01.439353 (+ 58us) server_negotiation.cc:299] Negotiation successful
0929 19:11:01.439366 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 19:11:01.439550 20338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:01.437227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:43477 (local address 127.9.236.3:60081)
0929 19:11:01.437443 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:01.437477 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:01.437927 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:01.438154 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:01.438157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:01.438166 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:11:01.439015 (+ 849us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:11:01.439015 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:11:01.439041 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:11:01.439067 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:11:01.439295 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:11:01.439297 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:11:01.439299 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:11:01.439355 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 19:11:01.439398 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":118,"threads_started":1}
I20260929 19:11:01.440138 20336 heartbeater.cc:347] Connected to a master server at 127.9.236.62:43477
I20260929 19:11:01.440222 20336 heartbeater.cc:464] Registering TS with master...
I20260929 19:11:01.440369 20336 heartbeater.cc:511] Master 127.9.236.62:43477 requested a full tablet report, sending...
I20260929 19:11:01.440806 19886 ts_manager.cc:194] Registered new tserver with Master: 683e8b2b172a4515b3530c69353b626a (127.9.236.3:44993)
I20260929 19:11:01.444723 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 20208
I20260929 19:11:01.444809 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:11:01.444989 10160 external_mini_cluster.cc:1428] Setting key bf702d2a12772f8fb7af612bd00426e3
I20260929 19:11:01.445971 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 19:11:01.447860 19925 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:11:01.447310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53666 (local address 127.9.236.62:43477)
0929 19:11:01.447385 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:01.447387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:01.447502 (+ 115us) server_negotiation.cc:407] Connection header received
0929 19:11:01.447569 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:01.447570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:01.447575 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:01.447618 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:01.447792 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53666: BlockingRecv error: recv got EOF from 127.0.0.1:53666 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 19:11:01.447943 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 19946
I20260929 19:11:01.452170 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 20077
I20260929 19:11:01.456359 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 20208
I20260929 19:11:01.460114 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 19855
2026-09-29T19:11:01Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (552 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:11:01 dist-test-slave-wftq krb5kdc[20346](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:11:01 dist-test-slave-wftq krb5kdc[20346](info): set up 2 sockets
Sep 29 19:11:01 dist-test-slave-wftq krb5kdc[20346](info): commencing operation
krb5kdc: starting...
W20260929 19:11:02.413949 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:11:02 dist-test-slave-wftq krb5kdc[20346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:11:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:11:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:11:02.536326 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:40683
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:43389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:40683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:11:02.623198 20359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:11:02.623432 20359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:11:02.623454 20359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:11:02.624588 20359 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:11:02.624622 20359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:11:02.624634 20359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:11:02.624645 20359 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:11:02.624656 20359 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:11:02.626216 20359 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:43389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:40683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:40683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:11:02.626421 20359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:11:02.626627 20359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:11:02.631656 20367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:02.631660 20365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:02.631661 20364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:02.631731 20359 server_base.cc:1034] running on GCE node
I20260929 19:11:02.632246 20359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:11:02.632548 20359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:11:02.633734 20359 hybrid_clock.cc:648] HybridClock initialized: now 1790709062633718 us; error 36 us; skew 500 ppm
Sep 29 19:11:02 dist-test-slave-wftq krb5kdc[20346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709062, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:11:02.636397 20359 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:11:02.636865 20359 webserver.cc:460] Webserver started at http://127.9.236.62:35991/ using document root <none> and password file <none>
I20260929 19:11:02.637077 20359 fs_manager.cc:359] Metadata directory not provided
I20260929 19:11:02.637164 20359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:11:02.637305 20359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:11:02.638410 20359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "b5689b420c8343a2906631685c5ccaa3"
format_stamp: "Formatted at 2026-09-29 19:11:02 on dist-test-slave-wftq"
server_key: "8dc417235f48be7d66b3a58a64055d0e"
server_key_iv: "2bfaf4c9608f412dcb25ce1e5c18924b"
server_key_version: "encryptionkey@0"
I20260929 19:11:02.638749 20359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "b5689b420c8343a2906631685c5ccaa3"
format_stamp: "Formatted at 2026-09-29 19:11:02 on dist-test-slave-wftq"
server_key: "8dc417235f48be7d66b3a58a64055d0e"
server_key_iv: "2bfaf4c9608f412dcb25ce1e5c18924b"
server_key_version: "encryptionkey@0"
I20260929 19:11:02.640084 20359 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:11:02.640868 20374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:02.641127 20359 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:11:02.641222 20359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "b5689b420c8343a2906631685c5ccaa3"
format_stamp: "Formatted at 2026-09-29 19:11:02 on dist-test-slave-wftq"
server_key: "8dc417235f48be7d66b3a58a64055d0e"
server_key_iv: "2bfaf4c9608f412dcb25ce1e5c18924b"
server_key_version: "encryptionkey@0"
I20260929 19:11:02.641289 20359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:11:02.662658 20359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:11:02.665431 20359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:11:02.665597 20359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:11:02.670704 20359 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:40683
I20260929 19:11:02.670761 20426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:40683 every 8 connection(s)
I20260929 19:11:02.671161 20359 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:11:02.671491 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 20359
I20260929 19:11:02.671604 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:11:02.671823 10160 external_mini_cluster.cc:1428] Setting key a7ee3d09756294574c998fa04e2f7724
I20260929 19:11:02.672317 20427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:11:02.675334 20427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3: Bootstrap starting.
I20260929 19:11:02.675875 20427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3: Neither blocks nor log segments found. Creating new log.
I20260929 19:11:02.676141 20427 log.cc:824] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3: Log is configured to *not* fsync() on all Append() calls
I20260929 19:11:02.676945 20427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3: No bootstrap required, opened a new log
Sep 29 19:11:02 dist-test-slave-wftq krb5kdc[20346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:11:02.679015 20427 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5689b420c8343a2906631685c5ccaa3" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40683 } }
I20260929 19:11:02.679193 20427 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:11:02.679246 20427 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5689b420c8343a2906631685c5ccaa3, State: Initialized, Role: FOLLOWER
I20260929 19:11:02.679384 20427 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5689b420c8343a2906631685c5ccaa3" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40683 } }
I20260929 19:11:02.679484 20427 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:11:02.679515 20427 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:11:02.679555 20427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:11:02.680224 20427 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5689b420c8343a2906631685c5ccaa3" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40683 } }
I20260929 19:11:02.680315 20427 leader_election.cc:302] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5689b420c8343a2906631685c5ccaa3; no voters:
I20260929 19:11:02.680476 20427 leader_election.cc:288] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:11:02.680729 20432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:11:02.680757 20427 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:11:02.680869 20432 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3 [term 1 LEADER]: Becoming Leader. State: Replica: b5689b420c8343a2906631685c5ccaa3, State: Running, Role: LEADER
I20260929 19:11:02.680964 20432 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5689b420c8343a2906631685c5ccaa3" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40683 } }
I20260929 19:11:02.681393 20434 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5689b420c8343a2906631685c5ccaa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5689b420c8343a2906631685c5ccaa3" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40683 } } }
I20260929 19:11:02.681524 20434 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3 [sys.catalog]: This master's current role is: LEADER
I20260929 19:11:02.682017 20433 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5689b420c8343a2906631685c5ccaa3. Latest consensus state: current_term: 1 leader_uuid: "b5689b420c8343a2906631685c5ccaa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5689b420c8343a2906631685c5ccaa3" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40683 } } }
I20260929 19:11:02.682222 20433 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3 [sys.catalog]: This master's current role is: LEADER
W20260929 19:11:02.682879 20448 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:11:02.682950 20448 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:11:02.683072 20441 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:11:02.683308 20441 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:11:02.683897 20430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:02.672415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56106 (local address 127.9.236.62:40683)
0929 19:11:02.672641 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:02.672651 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:02.672681 (+ 30us) server_negotiation.cc:407] Connection header received
0929 19:11:02.673286 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:02.673295 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:02.673352 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:02.673507 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:02.674193 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:02.675186 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:02.676062 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:02.676642 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:02.676795 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:02.679621 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:11:02.679626 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:11:02.679627 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:11:02.679642 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:11:02.681675 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:11:02.682271 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:11:02.682274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:11:02.682275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:11:02.682407 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:11:02.682683 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:11:02.682684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:11:02.682684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:11:02.683023 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:11:02.683170 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:02.683537 (+ 367us) server_negotiation.cc:299] Negotiation successful
0929 19:11:02.683687 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":107,"threads_started":1}
I20260929 19:11:02.684870 20441 catalog_manager.cc:1347] Generated new cluster ID: 826649742a88477fac11ac348a473370
I20260929 19:11:02.684916 20441 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:11:02.695025 20441 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:11:02.695853 20441 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:11:02.702744 20441 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3: Generated new TSK 0
I20260929 19:11:02.702983 20441 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.236.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
I20260929 19:11:02.733815 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:40683
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:43389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:11:02.842538 20453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:11:02.842737 20453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:11:02.842759 20453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:11:02.843910 20453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:11:02.843976 20453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:11:02.844024 20453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:11:02.845506 20453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:43389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:40683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:11:02.845724 20453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:11:02.845909 20453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:11:02.850936 20458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:02.850936 20461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:02.851049 20453 server_base.cc:1034] running on GCE node
W20260929 19:11:02.851023 20459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:02.851385 20453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:11:02.851603 20453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:11:02.852761 20453 hybrid_clock.cc:648] HybridClock initialized: now 1790709062852736 us; error 41 us; skew 500 ppm
Sep 29 19:11:02 dist-test-slave-wftq krb5kdc[20346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709062, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:11:02.855146 20453 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:11:02.855530 20453 webserver.cc:460] Webserver started at http://127.9.236.1:40965/ using document root <none> and password file <none>
I20260929 19:11:02.855764 20453 fs_manager.cc:359] Metadata directory not provided
I20260929 19:11:02.855809 20453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:11:02.855901 20453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:11:02.856845 20453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "39928704961a4b24913ef295cc1a1e96"
format_stamp: "Formatted at 2026-09-29 19:11:02 on dist-test-slave-wftq"
server_key: "3587474b1cf929a0a88a49b4b41d3529"
server_key_iv: "b60eab957bfbbcb501ea568557693722"
server_key_version: "encryptionkey@0"
I20260929 19:11:02.857161 20453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "39928704961a4b24913ef295cc1a1e96"
format_stamp: "Formatted at 2026-09-29 19:11:02 on dist-test-slave-wftq"
server_key: "3587474b1cf929a0a88a49b4b41d3529"
server_key_iv: "b60eab957bfbbcb501ea568557693722"
server_key_version: "encryptionkey@0"
I20260929 19:11:02.858438 20453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:11:02.859217 20468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:02.859438 20453 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:11:02.859539 20453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "39928704961a4b24913ef295cc1a1e96"
format_stamp: "Formatted at 2026-09-29 19:11:02 on dist-test-slave-wftq"
server_key: "3587474b1cf929a0a88a49b4b41d3529"
server_key_iv: "b60eab957bfbbcb501ea568557693722"
server_key_version: "encryptionkey@0"
I20260929 19:11:02.859616 20453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:11:02.878306 20453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:11:02.880460 20453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:11:02.880580 20453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:11:02.880780 20453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:11:02.881129 20453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:11:02.881171 20453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:02.881196 20453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:11:02.881222 20453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:02.887966 20453 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:45719
I20260929 19:11:02.888055 20581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:45719 every 8 connection(s)
I20260929 19:11:02.888348 20453 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:11:02.889520 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 20453
I20260929 19:11:02.889607 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:11:02.889787 10160 external_mini_cluster.cc:1428] Setting key 1fad6d6136d3038a82a0639e9e371f03
Sep 29 19:11:02 dist-test-slave-wftq krb5kdc[20346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709062, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.236.2@KRBTEST.COM; defaulting to no policy
I20260929 19:11:02.897884 20430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:02.889144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:49793 (local address 127.9.236.62:40683)
0929 19:11:02.889196 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:02.889198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:02.889447 (+ 249us) server_negotiation.cc:407] Connection header received
0929 19:11:02.890152 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:02.890153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:02.890166 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:02.890218 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:02.891176 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:02.891724 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:02.892713 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:02.892935 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:02.892989 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:02.895366 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:11:02.895368 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:11:02.895368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:11:02.895375 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:11:02.896607 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:11:02.896979 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:11:02.896980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:11:02.896980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:11:02.897028 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:11:02.897269 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:11:02.897270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:11:02.897270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:11:02.897376 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:11:02.897439 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:02.897694 (+ 255us) server_negotiation.cc:299] Negotiation successful
0929 19:11:02.897787 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:11:02.898069 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:02.889081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:40683 (local address 127.9.236.1:49793)
0929 19:11:02.889298 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:02.889337 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:02.890046 (+ 709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:02.890292 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:02.890296 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:02.890868 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:11:02.891088 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:02.891093 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:02.891803 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:02.891804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:02.892633 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:02.892636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:02.893000 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:02.893001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:02.893166 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:02.893675 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:11:02.893683 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:11:02.895283 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:11:02.896679 (+ 1396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:11:02.896686 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:11:02.896688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:11:02.896906 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:11:02.897080 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:11:02.897081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:11:02.897081 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:11:02.897213 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:11:02.897429 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:11:02.897434 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:11:02.897570 (+ 136us) client_negotiation.cc:769] Sending connection context
0929 19:11:02.897651 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 19:11:02.897802 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":104,"threads_started":1}
I20260929 19:11:02.898593 20582 heartbeater.cc:347] Connected to a master server at 127.9.236.62:40683
I20260929 19:11:02.898705 20582 heartbeater.cc:464] Registering TS with master...
I20260929 19:11:02.898921 20582 heartbeater.cc:511] Master 127.9.236.62:40683 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.2@KRBTEST.COM" created.
I20260929 19:11:02.899480 20391 ts_manager.cc:194] Registered new tserver with Master: 39928704961a4b24913ef295cc1a1e96 (127.9.236.1:45719)
I20260929 19:11:02.900084 20391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:49793
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
I20260929 19:11:02.914322 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:40683
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:43389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:11:02.994938 20587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:11:02.995155 20587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:11:02.995177 20587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:11:02.996321 20587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:11:02.996368 20587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:11:02.996393 20587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:11:02.997845 20587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:43389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:40683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:11:02.998070 20587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:11:02.998291 20587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:11:03.003513 20595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:03.003607 20593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:03.003525 20592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:03.003798 20587 server_base.cc:1034] running on GCE node
I20260929 19:11:03.003993 20587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:11:03.004220 20587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:11:03.005364 20587 hybrid_clock.cc:648] HybridClock initialized: now 1790709063005343 us; error 36 us; skew 500 ppm
Sep 29 19:11:03 dist-test-slave-wftq krb5kdc[20346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709063, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:11:03.007982 20587 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:11:03.008432 20587 webserver.cc:460] Webserver started at http://127.9.236.2:41801/ using document root <none> and password file <none>
I20260929 19:11:03.008628 20587 fs_manager.cc:359] Metadata directory not provided
I20260929 19:11:03.008675 20587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:11:03.008795 20587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:11:03.010084 20587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "33cdd9cc8c424a1789897b55115e80ad"
format_stamp: "Formatted at 2026-09-29 19:11:03 on dist-test-slave-wftq"
server_key: "534caec1076bb695bd1a0a272aef263d"
server_key_iv: "daba022fca843c420716137da9223107"
server_key_version: "encryptionkey@0"
I20260929 19:11:03.010458 20587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "33cdd9cc8c424a1789897b55115e80ad"
format_stamp: "Formatted at 2026-09-29 19:11:03 on dist-test-slave-wftq"
server_key: "534caec1076bb695bd1a0a272aef263d"
server_key_iv: "daba022fca843c420716137da9223107"
server_key_version: "encryptionkey@0"
I20260929 19:11:03.011746 20587 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 19:11:03.012461 20602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:03.012631 20587 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:11:03.012714 20587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "33cdd9cc8c424a1789897b55115e80ad"
format_stamp: "Formatted at 2026-09-29 19:11:03 on dist-test-slave-wftq"
server_key: "534caec1076bb695bd1a0a272aef263d"
server_key_iv: "daba022fca843c420716137da9223107"
server_key_version: "encryptionkey@0"
I20260929 19:11:03.012809 20587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:11:03.024998 20587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:11:03.026988 20587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:11:03.027122 20587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:11:03.027340 20587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:11:03.027643 20587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:11:03.027676 20587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:03.027714 20587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:11:03.027734 20587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:03.034077 20587 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:43829
I20260929 19:11:03.034158 20715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:43829 every 8 connection(s)
I20260929 19:11:03.034442 20587 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:11:03.039712 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 20587
I20260929 19:11:03.039810 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:11:03.039989 10160 external_mini_cluster.cc:1428] Setting key 796684eb2d419cbf9730200d00c50c17
Sep 29 19:11:03 dist-test-slave-wftq krb5kdc[20346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709063, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:11:03.044135 20430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:03.035219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:56303 (local address 127.9.236.62:40683)
0929 19:11:03.035322 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:03.035325 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:03.035540 (+ 215us) server_negotiation.cc:407] Connection header received
0929 19:11:03.036087 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:03.036089 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:03.036100 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:03.036162 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:03.037124 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.037699 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.038713 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.038914 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.039019 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:03.041423 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:11:03.041424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:11:03.041425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:11:03.041433 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:11:03.042818 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:11:03.043181 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:11:03.043183 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:11:03.043184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:11:03.043259 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:11:03.043505 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:11:03.043506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:11:03.043506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:11:03.043632 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:11:03.043693 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:03.043934 (+ 241us) server_negotiation.cc:299] Negotiation successful
0929 19:11:03.044036 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 19:11:03.044276 20718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:03.035188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:40683 (local address 127.9.236.2:56303)
0929 19:11:03.035391 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:03.035430 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:03.035973 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:03.036225 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:03.036228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:03.036781 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:11:03.036974 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:03.036978 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.037790 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.037791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:03.038613 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:03.038616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.039026 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.039027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:03.039122 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:03.039654 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:11:03.039662 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:11:03.041306 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:11:03.042884 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:11:03.042889 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:11:03.042891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:11:03.043090 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:11:03.043315 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:11:03.043316 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:11:03.043316 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:11:03.043449 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:11:03.043687 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:11:03.043692 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:11:03.043818 (+ 126us) client_negotiation.cc:769] Sending connection context
0929 19:11:03.043904 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 19:11:03.044049 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":123,"threads_started":1}
I20260929 19:11:03.044682 20716 heartbeater.cc:347] Connected to a master server at 127.9.236.62:40683
I20260929 19:11:03.044775 20716 heartbeater.cc:464] Registering TS with master...
I20260929 19:11:03.044926 20716 heartbeater.cc:511] Master 127.9.236.62:40683 requested a full tablet report, sending...
I20260929 19:11:03.045344 20391 ts_manager.cc:194] Registered new tserver with Master: 33cdd9cc8c424a1789897b55115e80ad (127.9.236.2:43829)
I20260929 19:11:03.045742 20391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:56303
WARNING: no policy specified for kudu/127.9.236.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
I20260929 19:11:03.066123 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:40683
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:43389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:11:03.152822 20721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:11:03.153053 20721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:11:03.153075 20721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:11:03.154333 20721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:11:03.154379 20721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:11:03.154404 20721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:11:03.155894 20721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:43389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:40683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:11:03.156137 20721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:11:03.156373 20721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:11:03.161509 20727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:03.161512 20726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:03.161702 20721 server_base.cc:1034] running on GCE node
W20260929 19:11:03.161512 20729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:03.162037 20721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:11:03.162297 20721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:11:03.163439 20721 hybrid_clock.cc:648] HybridClock initialized: now 1790709063163419 us; error 39 us; skew 500 ppm
Sep 29 19:11:03 dist-test-slave-wftq krb5kdc[20346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709063, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:11:03.166013 20721 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:11:03.166492 20721 webserver.cc:460] Webserver started at http://127.9.236.3:46791/ using document root <none> and password file <none>
I20260929 19:11:03.166721 20721 fs_manager.cc:359] Metadata directory not provided
I20260929 19:11:03.166770 20721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:11:03.166878 20721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:11:03.167948 20721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "95479cc1a9e0480d9fce7e1bb5ec557e"
format_stamp: "Formatted at 2026-09-29 19:11:03 on dist-test-slave-wftq"
server_key: "31bfdaadcfae848709c432d7cd2bef8c"
server_key_iv: "eb60a8befad181e51845c23632ab4f1a"
server_key_version: "encryptionkey@0"
I20260929 19:11:03.168265 20721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "95479cc1a9e0480d9fce7e1bb5ec557e"
format_stamp: "Formatted at 2026-09-29 19:11:03 on dist-test-slave-wftq"
server_key: "31bfdaadcfae848709c432d7cd2bef8c"
server_key_iv: "eb60a8befad181e51845c23632ab4f1a"
server_key_version: "encryptionkey@0"
I20260929 19:11:03.169597 20721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:11:03.170428 20736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:03.170622 20721 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:11:03.170706 20721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "95479cc1a9e0480d9fce7e1bb5ec557e"
format_stamp: "Formatted at 2026-09-29 19:11:03 on dist-test-slave-wftq"
server_key: "31bfdaadcfae848709c432d7cd2bef8c"
server_key_iv: "eb60a8befad181e51845c23632ab4f1a"
server_key_version: "encryptionkey@0"
I20260929 19:11:03.170778 20721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:11:03.180922 20721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:11:03.183012 20721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:11:03.183123 20721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:11:03.183324 20721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:11:03.183609 20721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:11:03.183656 20721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:03.183697 20721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:11:03.183727 20721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:03.189982 20721 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:41997
I20260929 19:11:03.190078 20849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:41997 every 8 connection(s)
I20260929 19:11:03.190352 20721 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:11:03.191725 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 20721
I20260929 19:11:03.191831 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:11:03.192044 10160 external_mini_cluster.cc:1428] Setting key 1b95f087e584aead23ee18fde701c5a6
Sep 29 19:11:03 dist-test-slave-wftq krb5kdc[20346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709063, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:11:03.200554 20430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:03.191123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:35239 (local address 127.9.236.62:40683)
0929 19:11:03.191233 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:03.191235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:03.191404 (+ 169us) server_negotiation.cc:407] Connection header received
0929 19:11:03.192208 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:03.192210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:03.192234 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:03.192313 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:03.193538 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.194042 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.195137 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.195330 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.195394 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:03.197624 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:11:03.197625 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:11:03.197625 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:11:03.197633 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:11:03.198991 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:11:03.199389 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:11:03.199390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:11:03.199391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:11:03.199446 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:11:03.199796 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:11:03.199798 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:11:03.199799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:11:03.199969 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:11:03.200048 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:03.200338 (+ 290us) server_negotiation.cc:299] Negotiation successful
0929 19:11:03.200443 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 19:11:03.200752 20852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:03.191030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:40683 (local address 127.9.236.3:35239)
0929 19:11:03.191255 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:03.191307 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:03.192073 (+ 766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:03.192448 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:03.192452 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:03.193216 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:11:03.193452 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:03.193458 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.194133 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.194133 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:03.195051 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:03.195053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.195406 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.195407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:03.195528 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:03.196031 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:11:03.196040 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:11:03.197510 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:11:03.199077 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:11:03.199083 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:11:03.199086 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:11:03.199316 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:11:03.199538 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:11:03.199539 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:11:03.199539 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:11:03.199686 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:11:03.200040 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:11:03.200046 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:11:03.200204 (+ 158us) client_negotiation.cc:769] Sending connection context
0929 19:11:03.200307 (+ 103us) client_negotiation.cc:240] Negotiation successful
0929 19:11:03.200503 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":93,"threads_started":1}
I20260929 19:11:03.201283 20850 heartbeater.cc:347] Connected to a master server at 127.9.236.62:40683
I20260929 19:11:03.201385 20850 heartbeater.cc:464] Registering TS with master...
I20260929 19:11:03.201582 20850 heartbeater.cc:511] Master 127.9.236.62:40683 requested a full tablet report, sending...
I20260929 19:11:03.202003 20391 ts_manager.cc:194] Registered new tserver with Master: 95479cc1a9e0480d9fce7e1bb5ec557e (127.9.236.3:41997)
I20260929 19:11:03.202446 20391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:35239
I20260929 19:11:03.203843 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:11:03.211184 20430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:03.205357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56110 (local address 127.9.236.62:40683)
0929 19:11:03.205426 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:03.205428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:03.205554 (+ 126us) server_negotiation.cc:407] Connection header received
0929 19:11:03.205622 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:03.205623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:03.205635 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:03.205690 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:03.206247 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.206764 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.207474 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.207678 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.207732 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:03.208869 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:11:03.208871 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:11:03.208871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:11:03.208878 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:11:03.210170 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:11:03.210464 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:11:03.210465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:11:03.210465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:11:03.210518 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:11:03.210673 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:11:03.210674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:11:03.210674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:11:03.210822 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:11:03.210921 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:03.211016 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 19:11:03.211090 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 19:11:03.212931 20391 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:11:03.214619 20391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:11:03.221431 20862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:03.217333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:43829 (local address 127.0.0.1:39668)
0929 19:11:03.217814 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:03.217823 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:03.217878 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:03.218154 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:03.218155 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:03.218157 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:11:03.218252 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:03.218254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.219120 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.219121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:03.220370 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:03.220373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.221096 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.221118 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:03.221204 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:03.221218 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 19:11:03.221284 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 19:11:03.221349 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":98,"threads_started":1}
I20260929 19:11:03.221444 20866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:03.217616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39668 (local address 127.9.236.2:43829)
0929 19:11:03.217809 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:03.217813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:03.217871 (+ 58us) server_negotiation.cc:407] Connection header received
0929 19:11:03.217983 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:03.217987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:03.218003 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:03.218067 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:11:03.218308 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.219025 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.220475 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.221014 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.221084 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:03.221175 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:03.221305 (+ 130us) server_negotiation.cc:299] Negotiation successful
0929 19:11:03.221350 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":89,"threads_started":1}
I20260929 19:11:03.221805 20861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:03.217332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:41997 (local address 127.0.0.1:52520)
0929 19:11:03.217670 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:03.217681 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:03.217724 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:03.218179 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:03.218180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:03.218182 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:11:03.218234 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:03.218235 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.219489 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.219490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:03.220408 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:03.220410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.221531 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.221532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:03.221605 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:03.221612 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 19:11:03.221671 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 19:11:03.221713 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":12,"thread_start_us":119,"threads_started":1}
I20260929 19:11:03.221876 20864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:03.217419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52674 (local address 127.9.236.1:45719)
0929 19:11:03.217617 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:03.217621 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:03.217632 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:11:03.217791 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:03.217793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:03.217809 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:03.217885 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:11:03.218122 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.219336 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.220671 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.221419 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.221472 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:03.221517 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:03.221759 (+ 242us) server_negotiation.cc:299] Negotiation successful
0929 19:11:03.221802 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":92,"threads_started":1}
I20260929 19:11:03.221992 20863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:03.217298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:45719 (local address 127.0.0.1:52674)
0929 19:11:03.217518 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:03.217557 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:03.217624 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:03.217916 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:03.217920 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:03.217923 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:11:03.218006 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:03.218008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.219457 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.219458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:03.220576 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:03.220577 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.221545 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.221546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:03.221605 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:03.221612 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 19:11:03.221658 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 19:11:03.221713 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":4,"thread_start_us":61,"threads_started":1}
I20260929 19:11:03.222154 20865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:03.217596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52520 (local address 127.9.236.3:41997)
0929 19:11:03.217797 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:03.217802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:03.217814 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:11:03.217859 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:03.217861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:03.217879 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:03.218098 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:11:03.218305 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.219389 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.220513 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.221349 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.221426 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:03.221489 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:03.222020 (+ 531us) server_negotiation.cc:299] Negotiation successful
0929 19:11:03.222074 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":90,"threads_started":1}
I20260929 19:11:03.222345 20650 tablet_service.cc:1467] Processing CreateTablet for tablet 3962f9b263364e1bb78598c0981f0070 (DEFAULT_TABLE table=test-table [id=b79a202d4d8241bbb4cebd589fc8a425]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:11:03.222780 20650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3962f9b263364e1bb78598c0981f0070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:11:03.222687 20516 tablet_service.cc:1467] Processing CreateTablet for tablet 3962f9b263364e1bb78598c0981f0070 (DEFAULT_TABLE table=test-table [id=b79a202d4d8241bbb4cebd589fc8a425]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:11:03.222824 20784 tablet_service.cc:1467] Processing CreateTablet for tablet 3962f9b263364e1bb78598c0981f0070 (DEFAULT_TABLE table=test-table [id=b79a202d4d8241bbb4cebd589fc8a425]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:11:03.223055 20516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3962f9b263364e1bb78598c0981f0070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:11:03.223136 20784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3962f9b263364e1bb78598c0981f0070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:11:03.225737 20867 tablet_bootstrap.cc:492] T 3962f9b263364e1bb78598c0981f0070 P 39928704961a4b24913ef295cc1a1e96: Bootstrap starting.
I20260929 19:11:03.226615 20867 tablet_bootstrap.cc:654] T 3962f9b263364e1bb78598c0981f0070 P 39928704961a4b24913ef295cc1a1e96: Neither blocks nor log segments found. Creating new log.
I20260929 19:11:03.226649 20869 tablet_bootstrap.cc:492] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e: Bootstrap starting.
I20260929 19:11:03.227284 20867 log.cc:824] T 3962f9b263364e1bb78598c0981f0070 P 39928704961a4b24913ef295cc1a1e96: Log is configured to *not* fsync() on all Append() calls
I20260929 19:11:03.227332 20869 tablet_bootstrap.cc:654] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e: Neither blocks nor log segments found. Creating new log.
I20260929 19:11:03.227736 20868 tablet_bootstrap.cc:492] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad: Bootstrap starting.
I20260929 19:11:03.227839 20869 log.cc:824] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e: Log is configured to *not* fsync() on all Append() calls
I20260929 19:11:03.228212 20867 tablet_bootstrap.cc:492] T 3962f9b263364e1bb78598c0981f0070 P 39928704961a4b24913ef295cc1a1e96: No bootstrap required, opened a new log
I20260929 19:11:03.228308 20867 ts_tablet_manager.cc:1397] T 3962f9b263364e1bb78598c0981f0070 P 39928704961a4b24913ef295cc1a1e96: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 19:11:03.228384 20868 tablet_bootstrap.cc:654] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad: Neither blocks nor log segments found. Creating new log.
I20260929 19:11:03.228652 20869 tablet_bootstrap.cc:492] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e: No bootstrap required, opened a new log
I20260929 19:11:03.228750 20869 ts_tablet_manager.cc:1397] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:11:03.228895 20868 log.cc:824] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad: Log is configured to *not* fsync() on all Append() calls
I20260929 19:11:03.229655 20868 tablet_bootstrap.cc:492] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad: No bootstrap required, opened a new log
I20260929 19:11:03.229776 20868 ts_tablet_manager.cc:1397] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:11:03.230603 20869 raft_consensus.cc:348] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39928704961a4b24913ef295cc1a1e96" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 45719 } } peers { permanent_uuid: "33cdd9cc8c424a1789897b55115e80ad" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43829 } } peers { permanent_uuid: "95479cc1a9e0480d9fce7e1bb5ec557e" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 41997 } }
I20260929 19:11:03.230758 20869 raft_consensus.cc:374] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:11:03.230783 20869 raft_consensus.cc:729] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95479cc1a9e0480d9fce7e1bb5ec557e, State: Initialized, Role: FOLLOWER
I20260929 19:11:03.230726 20867 raft_consensus.cc:348] T 3962f9b263364e1bb78598c0981f0070 P 39928704961a4b24913ef295cc1a1e96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39928704961a4b24913ef295cc1a1e96" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 45719 } } peers { permanent_uuid: "33cdd9cc8c424a1789897b55115e80ad" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43829 } } peers { permanent_uuid: "95479cc1a9e0480d9fce7e1bb5ec557e" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 41997 } }
I20260929 19:11:03.230917 20867 raft_consensus.cc:374] T 3962f9b263364e1bb78598c0981f0070 P 39928704961a4b24913ef295cc1a1e96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:11:03.230906 20869 consensus_queue.cc:260] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39928704961a4b24913ef295cc1a1e96" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 45719 } } peers { permanent_uuid: "33cdd9cc8c424a1789897b55115e80ad" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43829 } } peers { permanent_uuid: "95479cc1a9e0480d9fce7e1bb5ec557e" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 41997 } }
I20260929 19:11:03.230954 20867 raft_consensus.cc:729] T 3962f9b263364e1bb78598c0981f0070 P 39928704961a4b24913ef295cc1a1e96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39928704961a4b24913ef295cc1a1e96, State: Initialized, Role: FOLLOWER
I20260929 19:11:03.231174 20869 ts_tablet_manager.cc:1428] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 19:11:03.231204 20850 heartbeater.cc:503] Master 127.9.236.62:40683 was elected leader, sending a full tablet report...
I20260929 19:11:03.231118 20867 consensus_queue.cc:260] T 3962f9b263364e1bb78598c0981f0070 P 39928704961a4b24913ef295cc1a1e96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39928704961a4b24913ef295cc1a1e96" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 45719 } } peers { permanent_uuid: "33cdd9cc8c424a1789897b55115e80ad" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43829 } } peers { permanent_uuid: "95479cc1a9e0480d9fce7e1bb5ec557e" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 41997 } }
I20260929 19:11:03.231917 20867 ts_tablet_manager.cc:1428] T 3962f9b263364e1bb78598c0981f0070 P 39928704961a4b24913ef295cc1a1e96: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:11:03.232182 20582 heartbeater.cc:503] Master 127.9.236.62:40683 was elected leader, sending a full tablet report...
I20260929 19:11:03.232261 20868 raft_consensus.cc:348] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39928704961a4b24913ef295cc1a1e96" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 45719 } } peers { permanent_uuid: "33cdd9cc8c424a1789897b55115e80ad" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43829 } } peers { permanent_uuid: "95479cc1a9e0480d9fce7e1bb5ec557e" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 41997 } }
I20260929 19:11:03.232411 20868 raft_consensus.cc:374] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:11:03.232474 20868 raft_consensus.cc:729] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33cdd9cc8c424a1789897b55115e80ad, State: Initialized, Role: FOLLOWER
I20260929 19:11:03.232610 20868 consensus_queue.cc:260] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39928704961a4b24913ef295cc1a1e96" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 45719 } } peers { permanent_uuid: "33cdd9cc8c424a1789897b55115e80ad" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43829 } } peers { permanent_uuid: "95479cc1a9e0480d9fce7e1bb5ec557e" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 41997 } }
I20260929 19:11:03.232937 20868 ts_tablet_manager.cc:1428] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 19:11:03.233129 20716 heartbeater.cc:503] Master 127.9.236.62:40683 was elected leader, sending a full tablet report...
W20260929 19:11:03.285019 20717 tablet.cc:2369] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 19:11:03.389164 20583 tablet.cc:2369] T 3962f9b263364e1bb78598c0981f0070 P 39928704961a4b24913ef295cc1a1e96: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 19:11:03.440976 20851 tablet.cc:2369] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:11:03.581287 20873 raft_consensus.cc:482] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:11:03.581429 20873 raft_consensus.cc:504] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39928704961a4b24913ef295cc1a1e96" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 45719 } } peers { permanent_uuid: "33cdd9cc8c424a1789897b55115e80ad" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43829 } } peers { permanent_uuid: "95479cc1a9e0480d9fce7e1bb5ec557e" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 41997 } }
I20260929 19:11:03.581885 20873 leader_election.cc:288] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39928704961a4b24913ef295cc1a1e96 (127.9.236.1:45719), 33cdd9cc8c424a1789897b55115e80ad (127.9.236.2:43829)
I20260929 19:11:03.586241 20864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:03.582390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:60665 (local address 127.9.236.1:45719)
0929 19:11:03.582469 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:03.582471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:03.582482 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:11:03.582511 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:03.582513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:03.582526 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:03.582594 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:11:03.582789 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.583539 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.585186 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.585770 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.585853 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:03.585866 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:03.586128 (+ 262us) server_negotiation.cc:299] Negotiation successful
0929 19:11:03.586170 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 19:11:03.586241 20866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:03.582215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:60575 (local address 127.9.236.2:43829)
0929 19:11:03.582265 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:03.582267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:03.582506 (+ 239us) server_negotiation.cc:407] Connection header received
0929 19:11:03.582531 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:03.582533 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:03.582544 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:03.582595 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:11:03.582786 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.583536 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.585186 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.585770 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.585858 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:03.585866 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:03.586132 (+ 266us) server_negotiation.cc:299] Negotiation successful
0929 19:11:03.586171 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:11:03.586280 20852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:03.582071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:45719 (local address 127.9.236.3:60665)
0929 19:11:03.582209 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:03.582223 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:03.582283 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:03.582615 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:03.582616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:03.582618 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:11:03.582718 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:03.582720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.583633 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.583634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:03.585056 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:03.585060 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.585890 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.585892 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:03.586014 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:03.586028 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 19:11:03.586114 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 19:11:03.586168 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":66}
I20260929 19:11:03.586287 20876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:03.582174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:43829 (local address 127.9.236.3:60575)
0929 19:11:03.582377 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:03.582386 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:03.582448 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:03.582615 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:03.582616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:03.582618 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:11:03.582718 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:03.582720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.583633 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.583634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:03.585075 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:03.585076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.585930 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.585930 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:03.586014 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:03.586033 (+ 19us) client_negotiation.cc:769] Sending connection context
0929 19:11:03.586107 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 19:11:03.586168 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":111,"thread_start_us":125,"threads_started":1}
I20260929 19:11:03.586786 20670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3962f9b263364e1bb78598c0981f0070" candidate_uuid: "95479cc1a9e0480d9fce7e1bb5ec557e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33cdd9cc8c424a1789897b55115e80ad" is_pre_election: true
I20260929 19:11:03.586830 20536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3962f9b263364e1bb78598c0981f0070" candidate_uuid: "95479cc1a9e0480d9fce7e1bb5ec557e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39928704961a4b24913ef295cc1a1e96" is_pre_election: true
I20260929 19:11:03.586947 20670 raft_consensus.cc:2454] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95479cc1a9e0480d9fce7e1bb5ec557e in term 0.
I20260929 19:11:03.587049 20536 raft_consensus.cc:2454] T 3962f9b263364e1bb78598c0981f0070 P 39928704961a4b24913ef295cc1a1e96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95479cc1a9e0480d9fce7e1bb5ec557e in term 0.
I20260929 19:11:03.587296 20739 leader_election.cc:302] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33cdd9cc8c424a1789897b55115e80ad, 95479cc1a9e0480d9fce7e1bb5ec557e; no voters:
I20260929 19:11:03.587538 20873 raft_consensus.cc:2781] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:11:03.587587 20873 raft_consensus.cc:482] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:11:03.587605 20873 raft_consensus.cc:3037] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:11:03.588479 20873 raft_consensus.cc:504] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39928704961a4b24913ef295cc1a1e96" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 45719 } } peers { permanent_uuid: "33cdd9cc8c424a1789897b55115e80ad" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43829 } } peers { permanent_uuid: "95479cc1a9e0480d9fce7e1bb5ec557e" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 41997 } }
I20260929 19:11:03.588635 20873 leader_election.cc:288] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e [CANDIDATE]: Term 1 election: Requested vote from peers 39928704961a4b24913ef295cc1a1e96 (127.9.236.1:45719), 33cdd9cc8c424a1789897b55115e80ad (127.9.236.2:43829)
I20260929 19:11:03.588865 20670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3962f9b263364e1bb78598c0981f0070" candidate_uuid: "95479cc1a9e0480d9fce7e1bb5ec557e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33cdd9cc8c424a1789897b55115e80ad"
I20260929 19:11:03.588827 20536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3962f9b263364e1bb78598c0981f0070" candidate_uuid: "95479cc1a9e0480d9fce7e1bb5ec557e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39928704961a4b24913ef295cc1a1e96"
I20260929 19:11:03.588940 20670 raft_consensus.cc:3037] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:11:03.588941 20536 raft_consensus.cc:3037] T 3962f9b263364e1bb78598c0981f0070 P 39928704961a4b24913ef295cc1a1e96 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:11:03.589888 20670 raft_consensus.cc:2454] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95479cc1a9e0480d9fce7e1bb5ec557e in term 1.
I20260929 19:11:03.589884 20536 raft_consensus.cc:2454] T 3962f9b263364e1bb78598c0981f0070 P 39928704961a4b24913ef295cc1a1e96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95479cc1a9e0480d9fce7e1bb5ec557e in term 1.
I20260929 19:11:03.590096 20739 leader_election.cc:302] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33cdd9cc8c424a1789897b55115e80ad, 95479cc1a9e0480d9fce7e1bb5ec557e; no voters:
I20260929 19:11:03.590237 20873 raft_consensus.cc:2781] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:11:03.590409 20873 raft_consensus.cc:686] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e [term 1 LEADER]: Becoming Leader. State: Replica: 95479cc1a9e0480d9fce7e1bb5ec557e, State: Running, Role: LEADER
I20260929 19:11:03.590540 20873 consensus_queue.cc:237] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39928704961a4b24913ef295cc1a1e96" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 45719 } } peers { permanent_uuid: "33cdd9cc8c424a1789897b55115e80ad" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43829 } } peers { permanent_uuid: "95479cc1a9e0480d9fce7e1bb5ec557e" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 41997 } }
I20260929 19:11:03.591498 20390 catalog_manager.cc:5495] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e reported cstate change: term changed from 0 to 1, leader changed from <none> to 95479cc1a9e0480d9fce7e1bb5ec557e (127.9.236.3). New cstate: current_term: 1 leader_uuid: "95479cc1a9e0480d9fce7e1bb5ec557e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39928704961a4b24913ef295cc1a1e96" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 45719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33cdd9cc8c424a1789897b55115e80ad" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95479cc1a9e0480d9fce7e1bb5ec557e" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 41997 } health_report { overall_health: HEALTHY } } }
I20260929 19:11:03.662758 20865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:03.660291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52526 (local address 127.9.236.3:41997)
0929 19:11:03.660365 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:03.660367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:03.660376 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:11:03.660412 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:03.660412 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:03.660434 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:03.660501 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:11:03.660665 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.661165 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.661978 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:03.662181 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:03.662236 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:03.662372 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:11:03.662477 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:11:03.662542 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:03.662646 (+ 104us) server_negotiation.cc:299] Negotiation successful
0929 19:11:03.662665 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 19:11:03.664808 20670 raft_consensus.cc:1261] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad [term 1 FOLLOWER]: Refusing update from remote peer 95479cc1a9e0480d9fce7e1bb5ec557e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:11:03.664849 20536 raft_consensus.cc:1261] T 3962f9b263364e1bb78598c0981f0070 P 39928704961a4b24913ef295cc1a1e96 [term 1 FOLLOWER]: Refusing update from remote peer 95479cc1a9e0480d9fce7e1bb5ec557e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:11:03.665186 20877 consensus_queue.cc:1035] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e [LEADER]: Connected to new peer: Peer: permanent_uuid: "33cdd9cc8c424a1789897b55115e80ad" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:11:03.665369 20873 consensus_queue.cc:1035] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e [LEADER]: Connected to new peer: Peer: permanent_uuid: "39928704961a4b24913ef295cc1a1e96" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 45719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:11:03.668007 20881 mvcc.cc:205] Tried to move back new op lower bound from 7334744324768321536 to 7334744324467539968. Current Snapshot: MvccSnapshot[applied={T|T < 7334744324768321536}]
I20260929 19:11:03.668677 20882 mvcc.cc:205] Tried to move back new op lower bound from 7334744324768321536 to 7334744324467539968. Current Snapshot: MvccSnapshot[applied={T|T < 7334744324768321536}]
I20260929 19:11:03.670979 20390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:11:03.671111 20390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:11:03.671960 20390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5689b420c8343a2906631685c5ccaa3: Sending DeleteTablet for 3 replicas of tablet 3962f9b263364e1bb78598c0981f0070
I20260929 19:11:03.672434 20650 tablet_service.cc:1514] Processing DeleteTablet for tablet 3962f9b263364e1bb78598c0981f0070 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:11:03 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:39668
I20260929 19:11:03.672442 20516 tablet_service.cc:1514] Processing DeleteTablet for tablet 3962f9b263364e1bb78598c0981f0070 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:11:03 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:52674
I20260929 19:11:03.672696 20784 tablet_service.cc:1514] Processing DeleteTablet for tablet 3962f9b263364e1bb78598c0981f0070 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:11:03 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:52520
I20260929 19:11:03.672914 20891 tablet_replica.cc:330] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e: stopping tablet replica
I20260929 19:11:03.673009 20890 tablet_replica.cc:330] T 3962f9b263364e1bb78598c0981f0070 P 39928704961a4b24913ef295cc1a1e96: stopping tablet replica
I20260929 19:11:03.673051 20891 raft_consensus.cc:2229] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:11:03.673177 20890 raft_consensus.cc:2229] T 3962f9b263364e1bb78598c0981f0070 P 39928704961a4b24913ef295cc1a1e96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:11:03.673327 20891 raft_consensus.cc:2258] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:11:03.673400 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 20453
I20260929 19:11:03.675328 20891 ts_tablet_manager.cc:1905] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:11:03.677121 20891 ts_tablet_manager.cc:1918] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:11:03.677201 20891 log.cc:1196] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/3962f9b263364e1bb78598c0981f0070
I20260929 19:11:03.677426 20889 tablet_replica.cc:330] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad: stopping tablet replica
I20260929 19:11:03.677467 20891 ts_tablet_manager.cc:1939] T 3962f9b263364e1bb78598c0981f0070 P 95479cc1a9e0480d9fce7e1bb5ec557e: Deleting consensus metadata
I20260929 19:11:03.677663 20889 raft_consensus.cc:2229] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:11:03.677855 20889 raft_consensus.cc:2258] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:11:03.678155 20375 catalog_manager.cc:4841] TS 95479cc1a9e0480d9fce7e1bb5ec557e (127.9.236.3:41997): tablet 3962f9b263364e1bb78598c0981f0070 (table test-table [id=b79a202d4d8241bbb4cebd589fc8a425]) successfully deleted
I20260929 19:11:03.678601 20889 ts_tablet_manager.cc:1905] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 19:11:03.679410 20376 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.236.1:45719 (error 108)
W20260929 19:11:03.680080 20863 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:11:03.679915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:45719 (local address 127.0.0.1:52686)
0929 19:11:03.679982 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:03.680030 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.1:45719: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":45}
W20260929 19:11:03.680210 20376 catalog_manager.cc:4584] TS 39928704961a4b24913ef295cc1a1e96 (127.9.236.1:45719): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3962f9b263364e1bb78598c0981f0070: Network error: Client connection negotiation failed: client connection to 127.9.236.1:45719: connect: Connection reset by peer (error 104)
I20260929 19:11:03.680401 20889 ts_tablet_manager.cc:1918] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:11:03.680475 20889 log.cc:1196] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/3962f9b263364e1bb78598c0981f0070
I20260929 19:11:03.680583 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 20587
I20260929 19:11:03.680728 20889 ts_tablet_manager.cc:1939] T 3962f9b263364e1bb78598c0981f0070 P 33cdd9cc8c424a1789897b55115e80ad: Deleting consensus metadata
I20260929 19:11:03.685930 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 20721
I20260929 19:11:03.685988 20863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:03.685815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:43829 (local address 127.0.0.1:39670)
0929 19:11:03.685902 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:03.685948 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.2:43829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260929 19:11:03.691545 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 20359
2026-09-29T19:11:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2233 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:11:03 dist-test-slave-wftq krb5kdc[20895](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:11:03 dist-test-slave-wftq krb5kdc[20895](info): set up 2 sockets
Sep 29 19:11:03 dist-test-slave-wftq krb5kdc[20895](info): commencing operation
krb5kdc: starting...
W20260929 19:11:04.641475 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:11:04 dist-test-slave-wftq krb5kdc[20895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:11:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:11:04Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.236.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.236.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.236.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/rangeradmin_127.9.236.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.236.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/rangeradmin_127.9.236.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.236.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.236.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.236.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/rangerlookup_127.9.236.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.236.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/rangerlookup_127.9.236.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.236.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.236.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.236.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/HTTP_127.9.236.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.236.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/HTTP_127.9.236.20@KRBTEST.COM.keytab.
I20260929 19:11:04.803339 10160 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task0UA8iT/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/postgres -l logfile start
2026-09-29 19:11:07.253 UTC [20925] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 19:11:07.254 UTC [20925] LOG: listening on IPv4 address "127.9.236.20", port 49049
2026-09-29 19:11:07.257 UTC [20925] LOG: listening on Unix socket "/tmp/.s.PGSQL.49049"
2026-09-29 19:11:07.288 UTC [20928] LOG: database system was shut down at 2026-09-29 19:11:05 UTC
2026-09-29 19:11:07.292 UTC [20925] LOG: database system is ready to accept connections
I20260929 19:11:08.162045 10160 mini_postgres.cc:96] Postgres bound to 49049
2026-09-29 19:11:08.168 UTC [20936] FATAL: database "slave" does not exist
127.9.236.20:49049 - accepting connections
I20260929 19:11:08.169126 10160 mini_ranger.cc:161] Starting Ranger...
I20260929 19:11:08.185050 10160 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 19:11:08.648382 10160 mini_ranger.cc:87] Created ranger Postgres database
I20260929 19:11:08.648454 10160 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ranger-admin
2026-09-29 19:11:08,919 [I] DB FLAVOR :POSTGRES
2026-09-29 19:11:08,920 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 19:11:08,920 [I] Checking connection..
2026-09-29 19:11:08,920 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 19:11:09,191 [I] Checking connection passed.
2026-09-29 19:11:09,192 [I] --------- Verifying version history table ---------
2026-09-29 19:11:09,192 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 19:11:09,448 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 19:11:09,448 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 19:11:09,719 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 19:11:09,719 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 19:11:09,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 19:11:09.971 UTC [21000] WARNING: there is no transaction in progress
2026-09-29 19:11:09,978 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 19:11:09,979 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 19:11:10,243 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 19:11:10,244 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 19:11:10,244 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 19:11:10,500 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 19:11:10,744 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-wftq.c.gcp-upstream.internal','N') ;"
2026-09-29 19:11:11,004 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 19:11:11,267 [I] Table x_portal_user does not exist in database ranger
2026-09-29 19:11:11,267 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 19:11:11,539 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 19:11:11,539 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 19:11:11,799 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 19:11:12,046 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 19:11:12,046 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 19:11:13.016 UTC [21108] WARNING: there is no transaction in progress
2026-09-29 19:11:13.033 UTC [21108] WARNING: there is no transaction in progress
2026-09-29 19:11:13.051 UTC [21108] WARNING: there is no transaction in progress
2026-09-29 19:11:13.068 UTC [21108] WARNING: there is no transaction in progress
2026-09-29 19:11:13.086 UTC [21108] WARNING: there is no transaction in progress
2026-09-29 19:11:13.104 UTC [21108] WARNING: there is no transaction in progress
2026-09-29 19:11:13.160 UTC [21108] WARNING: there is no transaction in progress
2026-09-29 19:11:13.167 UTC [21108] WARNING: there is no transaction in progress
2026-09-29 19:11:13.181 UTC [21108] WARNING: there is no transaction in progress
2026-09-29 19:11:13.192 UTC [21108] WARNING: there is no transaction in progress
2026-09-29 19:11:13.203 UTC [21108] WARNING: there is no transaction in progress
2026-09-29 19:11:13.218 UTC [21108] WARNING: there is no transaction in progress
2026-09-29 19:11:13.229 UTC [21108] WARNING: there is no transaction in progress
2026-09-29 19:11:13.236 UTC [21108] WARNING: there is no transaction in progress
2026-09-29 19:11:13.245 UTC [21108] WARNING: there is no transaction in progress
2026-09-29 19:11:13.252 UTC [21108] WARNING: there is no transaction in progress
2026-09-29 19:11:13,896 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 19:11:13,896 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 19:11:14,146 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 19:11:14,146 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 19:11:14,419 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 19:11:14,419 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 19:11:14,682 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 19:11:14,682 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 19:11:14,948 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 19:11:15,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-wftq.c.gcp-upstream.internal';"
2026-09-29 19:11:15,474 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 19:11:15,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 19:11:15,726 [I] DB_PATCHES have already been applied
I20260929 19:11:15.732960 10160 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ranger-admin:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task0UA8iT/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 19:11:15.733038 10160 mini_ranger.cc:193] Using host: 127.9.236.20
I20260929 19:11:15.736368 10160 mini_ranger.cc:227] Ranger admin URL: http://127.9.236.20:39099
Sep 29, 2026 7:11:16 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 7:11:16 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 7:11:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 7:11:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 7:11:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 7:11:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 7:11:16 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 19:11:16.736328 10160 mini_ranger.cc:160] Time spent starting Ranger: real 8.567s user 0.002s sys 0.008s
Sep 29, 2026 7:11:36 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 7:11:36 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 29, 2026 7:11:36 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 7:11:36 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 7:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 7:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 7:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 7:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 7:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 7:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 7:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 7:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 7:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 7:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 7:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 7:11:36 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 7:11:36 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 7:11:37 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 19:11:38.024302 10160 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.236.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.236.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.236.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/rangerkms_127.9.236.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.236.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/rangerkms_127.9.236.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.236.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.236.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.236.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/HTTP_127.9.236.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.236.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/HTTP_127.9.236.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 19:11:38 dist-test-slave-wftq krb5kdc[20895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709098, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 19:11:38.100521 10160 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 19:11:38.110163 10160 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 19:11:38.968657 10160 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 19:11:38.968737 10160 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ranger-kms
I20260929 19:11:38.968755 10160 mini_ranger_kms.cc:227] Using postgres at 127.9.236.20:49049
2026-09-29 19:11:39,125 [I] DB FLAVOR :POSTGRES
2026-09-29 19:11:39,125 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 19:11:39,125 [I] Checking connection
2026-09-29 19:11:39,125 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 19:11:39,379 [I] connection success
2026-09-29 19:11:39,380 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 19:11:39,380 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 19:11:39,380 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 19:11:39,610 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 19:11:39,611 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 19:11:39,611 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 19:11:39,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:49049/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 19:11:39,868 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 19:11:40.301285 10160 mini_ranger_kms.cc:310] Created kms service
I20260929 19:11:40.509172 10160 mini_ranger_kms.cc:326] Created kudu user
I20260929 19:11:40.552062 10160 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 19:11:40 dist-test-slave-wftq krb5kdc[20895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709098, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.236.20@KRBTEST.COM
I20260929 19:11:40.979820 10160 mini_ranger_kms.cc:384] Added ranger policy
I20260929 19:11:40.979925 10160 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ranger-kms:/tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 19:11:40.979945 10160 mini_ranger_kms.cc:240] Using host: 127.9.236.20
I20260929 19:11:40.982961 10160 mini_ranger_kms.cc:276] Ranger KMS PID: 21288
I20260929 19:11:40.983067 10160 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.236.20:46663
Sep 29, 2026 7:11:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 7:11:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 7:11:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 7:11:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 7:11:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 7:11:42 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 19:11:43.101255 10160 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.001s user 0.010s sys 0.127s
I20260929 19:11:43.101387 10160 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 19:11:43.101483 10160 mini_ranger_kms.cc:401] 127.9.236.20:46663/kms/v1/keys
Sep 29 19:11:43 dist-test-slave-wftq krb5kdc[20895](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790709103, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.236.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 19:11:43 dist-test-slave-wftq krb5kdc[20895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709103, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.236.20@KRBTEST.COM for HTTP/127.9.236.20@KRBTEST.COM
Sep 29 19:11:44 dist-test-slave-wftq krb5kdc[20895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709103, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.236.20@KRBTEST.COM for HTTP/127.9.236.20@KRBTEST.COM
Sep 29 19:11:46 dist-test-slave-wftq krb5kdc[20895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:11:46.583948 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:42515
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:35159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:42515
--ranger_config_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.236.20:46663/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:11:46.703051 21339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:11:46.703287 21339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:11:46.703308 21339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:11:46.704470 21339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:11:46.704511 21339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:11:46.704524 21339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:11:46.704535 21339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:11:46.704545 21339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:11:46.706183 21339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:35159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.236.20:46663/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:42515
--ranger_config_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:42515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:11:46.706398 21339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:11:46.706599 21339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:11:46.712181 21345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:46.712240 21344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:46.712348 21347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:46.712399 21339 server_base.cc:1034] running on GCE node
I20260929 19:11:46.712749 21339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:11:46.713256 21339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:11:46.714437 21339 hybrid_clock.cc:648] HybridClock initialized: now 1790709106714398 us; error 64 us; skew 500 ppm
Sep 29 19:11:46 dist-test-slave-wftq krb5kdc[20895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709106, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:11:46.722172 21339 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:11:46.722718 21339 webserver.cc:460] Webserver started at http://127.9.236.62:34459/ using document root <none> and password file <none>
I20260929 19:11:46.722958 21339 fs_manager.cc:359] Metadata directory not provided
I20260929 19:11:46.723002 21339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:11:46.723133 21339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 19:11:46 dist-test-slave-wftq krb5kdc[20895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709106, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for HTTP/127.9.236.20@KRBTEST.COM
I20260929 19:11:46.776320 21339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "2a048bddb4fe42298bc2d1a9ed9d7f2d"
format_stamp: "Formatted at 2026-09-29 19:11:46 on dist-test-slave-wftq"
server_key: "d24b7f571d9055ab21dd0731d977aac9"
server_key_iv: "62f3aaf4bc13e77627ba3f80a9cc73bc"
server_key_version: "kuduclusterkey@0"
I20260929 19:11:46.776758 21339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "2a048bddb4fe42298bc2d1a9ed9d7f2d"
format_stamp: "Formatted at 2026-09-29 19:11:46 on dist-test-slave-wftq"
server_key: "d24b7f571d9055ab21dd0731d977aac9"
server_key_iv: "62f3aaf4bc13e77627ba3f80a9cc73bc"
server_key_version: "kuduclusterkey@0"
I20260929 19:11:46.778318 21339 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260929 19:11:46.795936 21354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:46.796170 21339 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:11:46.796269 21339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "2a048bddb4fe42298bc2d1a9ed9d7f2d"
format_stamp: "Formatted at 2026-09-29 19:11:46 on dist-test-slave-wftq"
server_key: "d24b7f571d9055ab21dd0731d977aac9"
server_key_iv: "62f3aaf4bc13e77627ba3f80a9cc73bc"
server_key_version: "kuduclusterkey@0"
I20260929 19:11:46.796378 21339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:11:46.810948 21339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:11:46.812862 21339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:11:46.813004 21339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:11:46.818444 21339 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:42515
I20260929 19:11:46.818617 21406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:42515 every 8 connection(s)
I20260929 19:11:46.818897 21339 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:11:46.819806 21407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:11:46.821293 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 21339
I20260929 19:11:46.821399 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:11:46.821650 10160 external_mini_cluster.cc:1428] Setting key f861557d37ba7f810bf72d1bf35d80e3
I20260929 19:11:46.824481 21407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d: Bootstrap starting.
I20260929 19:11:46.825321 21407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d: Neither blocks nor log segments found. Creating new log.
I20260929 19:11:46.825631 21407 log.cc:824] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d: Log is configured to *not* fsync() on all Append() calls
I20260929 19:11:46.826578 21407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d: No bootstrap required, opened a new log
Sep 29 19:11:46 dist-test-slave-wftq krb5kdc[20895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:11:46.829228 21407 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a048bddb4fe42298bc2d1a9ed9d7f2d" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 42515 } }
I20260929 19:11:46.829424 21407 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:11:46.829509 21407 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a048bddb4fe42298bc2d1a9ed9d7f2d, State: Initialized, Role: FOLLOWER
I20260929 19:11:46.829653 21407 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a048bddb4fe42298bc2d1a9ed9d7f2d" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 42515 } }
I20260929 19:11:46.829730 21407 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:11:46.829759 21407 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:11:46.829831 21407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:11:46.830579 21407 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a048bddb4fe42298bc2d1a9ed9d7f2d" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 42515 } }
I20260929 19:11:46.830703 21407 leader_election.cc:302] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a048bddb4fe42298bc2d1a9ed9d7f2d; no voters:
I20260929 19:11:46.830951 21407 leader_election.cc:288] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:11:46.830994 21412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:11:46.831383 21407 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:11:46.831418 21412 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d [term 1 LEADER]: Becoming Leader. State: Replica: 2a048bddb4fe42298bc2d1a9ed9d7f2d, State: Running, Role: LEADER
I20260929 19:11:46.831610 21412 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a048bddb4fe42298bc2d1a9ed9d7f2d" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 42515 } }
I20260929 19:11:46.831940 21412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a048bddb4fe42298bc2d1a9ed9d7f2d. Latest consensus state: current_term: 1 leader_uuid: "2a048bddb4fe42298bc2d1a9ed9d7f2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a048bddb4fe42298bc2d1a9ed9d7f2d" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 42515 } } }
I20260929 19:11:46.831996 21412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d [sys.catalog]: This master's current role is: LEADER
I20260929 19:11:46.832207 21416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:11:46.832243 21410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:46.822175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42366 (local address 127.9.236.62:42515)
0929 19:11:46.822400 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:46.822411 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:46.822438 (+ 27us) server_negotiation.cc:407] Connection header received
0929 19:11:46.822976 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:46.822984 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:46.823040 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:46.823177 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:46.823811 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:46.824792 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:46.825678 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:46.826280 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:46.826386 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:46.828635 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:11:46.828640 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:11:46.828640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:11:46.828651 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:11:46.830382 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:11:46.830735 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:11:46.830738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:11:46.830739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:11:46.830838 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:11:46.831026 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:11:46.831027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:11:46.831028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:11:46.831328 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:11:46.831493 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:46.831930 (+ 437us) server_negotiation.cc:299] Negotiation successful
0929 19:11:46.832043 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":112,"threads_started":1}
I20260929 19:11:46.832829 21416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:11:46.832345 21414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a048bddb4fe42298bc2d1a9ed9d7f2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a048bddb4fe42298bc2d1a9ed9d7f2d" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 42515 } } }
I20260929 19:11:46.835521 21414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d [sys.catalog]: This master's current role is: LEADER
I20260929 19:11:46.836050 21407 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 19:11:46.840927 21416 catalog_manager.cc:1347] Generated new cluster ID: 03fab14358a64bb290d1e5738154dd7f
I20260929 19:11:46.841173 21416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:11:46.851727 21416 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:11:46.852608 21416 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:11:46.862239 21416 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d: Generated new TSK 0
I20260929 19:11:46.862517 21416 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.236.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
I20260929 19:11:47.603996 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.236.20:46663/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:42515
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:35159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:11:47.737489 21452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:11:47.737852 21452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:11:47.737892 21452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:11:47.739919 21452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:11:47.740005 21452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:11:47.740053 21452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:11:47.742568 21452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:35159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.236.20:46663/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:42515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:11:47.742909 21452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:11:47.743239 21452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:11:47.749328 21457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:47.749289 21460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:47.749292 21458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:47.750136 21452 server_base.cc:1034] running on GCE node
I20260929 19:11:47.750344 21452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:11:47.750603 21452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:11:47.753825 21452 hybrid_clock.cc:648] HybridClock initialized: now 1790709107753795 us; error 59 us; skew 500 ppm
Sep 29 19:11:47 dist-test-slave-wftq krb5kdc[20895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709107, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:11:47.756199 21452 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:11:47.756651 21452 webserver.cc:460] Webserver started at http://127.9.236.1:40659/ using document root <none> and password file <none>
I20260929 19:11:47.756861 21452 fs_manager.cc:359] Metadata directory not provided
I20260929 19:11:47.756907 21452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:11:47.757043 21452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 19:11:47 dist-test-slave-wftq krb5kdc[20895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709107, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for HTTP/127.9.236.20@KRBTEST.COM
I20260929 19:11:47.771118 21452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "62cd101d6f624d478b8fbca5f1736ad6"
format_stamp: "Formatted at 2026-09-29 19:11:47 on dist-test-slave-wftq"
server_key: "f204337b97390122ba614b09e31a7956"
server_key_iv: "e58c84d859464327ca0aded7465ee937"
server_key_version: "kuduclusterkey@0"
I20260929 19:11:47.771848 21452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "62cd101d6f624d478b8fbca5f1736ad6"
format_stamp: "Formatted at 2026-09-29 19:11:47 on dist-test-slave-wftq"
server_key: "f204337b97390122ba614b09e31a7956"
server_key_iv: "e58c84d859464327ca0aded7465ee937"
server_key_version: "kuduclusterkey@0"
I20260929 19:11:47.773976 21452 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260929 19:11:47.789532 21467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:47.789717 21452 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:11:47.789812 21452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "62cd101d6f624d478b8fbca5f1736ad6"
format_stamp: "Formatted at 2026-09-29 19:11:47 on dist-test-slave-wftq"
server_key: "f204337b97390122ba614b09e31a7956"
server_key_iv: "e58c84d859464327ca0aded7465ee937"
server_key_version: "kuduclusterkey@0"
I20260929 19:11:47.789916 21452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:11:47.805819 21452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:11:47.807752 21452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:11:47.807890 21452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:11:47.808149 21452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:11:47.808604 21452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:11:47.808706 21452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:47.808815 21452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:11:47.808892 21452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:47.817117 21452 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:39773
I20260929 19:11:47.817728 21452 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:11:47.819082 21581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:39773 every 8 connection(s)
I20260929 19:11:47.828521 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 21452
I20260929 19:11:47.828645 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:11:47.828881 10160 external_mini_cluster.cc:1428] Setting key d82e1951bd132b08904b6123c930537c
Sep 29 19:11:47 dist-test-slave-wftq krb5kdc[20895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709107, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.236.2@KRBTEST.COM; defaulting to no policy
I20260929 19:11:47.842890 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:47.819766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:42515 (local address 127.9.236.1:51669)
0929 19:11:47.819938 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:47.819983 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:47.820592 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:47.820966 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:47.820970 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:47.821225 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:11:47.821532 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:47.821539 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:47.831964 (+ 10425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:47.831966 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:47.833461 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:47.833466 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:47.833988 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:47.833989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:47.834150 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:47.834846 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:11:47.834868 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:11:47.836662 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:11:47.840891 (+ 4229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:11:47.840902 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:11:47.840905 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:11:47.841277 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:11:47.841658 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:11:47.841659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:11:47.841659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:11:47.841794 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:11:47.842080 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:11:47.842086 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:11:47.842278 (+ 192us) client_negotiation.cc:769] Sending connection context
0929 19:11:47.842452 (+ 174us) client_negotiation.cc:240] Negotiation successful
0929 19:11:47.842634 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.2@KRBTEST.COM" created.
I20260929 19:11:47.845396 21585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:47.820100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:51669 (local address 127.9.236.62:42515)
0929 19:11:47.820749 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:47.820750 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:47.820757 (+ 7us) server_negotiation.cc:407] Connection header received
0929 19:11:47.820780 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:47.820781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:47.820792 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:47.820860 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:47.822582 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:47.831780 (+ 9198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:47.833576 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:47.833897 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:47.835615 (+ 1718us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:47.836796 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:11:47.836798 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:11:47.836798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:11:47.836808 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:11:47.840771 (+ 3963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:11:47.841520 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:11:47.841521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:11:47.841521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:11:47.841589 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:11:47.841856 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:11:47.841857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:11:47.841857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:11:47.842009 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:11:47.845152 (+ 3143us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:47.845197 (+ 45us) server_negotiation.cc:299] Negotiation successful
0929 19:11:47.845294 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":89,"threads_started":1}
I20260929 19:11:47.845969 21582 heartbeater.cc:347] Connected to a master server at 127.9.236.62:42515
I20260929 19:11:47.846068 21582 heartbeater.cc:464] Registering TS with master...
I20260929 19:11:47.846271 21582 heartbeater.cc:511] Master 127.9.236.62:42515 requested a full tablet report, sending...
I20260929 19:11:47.846881 21371 ts_manager.cc:194] Registered new tserver with Master: 62cd101d6f624d478b8fbca5f1736ad6 (127.9.236.1:39773)
I20260929 19:11:47.847764 21371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:51669
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
I20260929 19:11:47.864038 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.236.20:46663/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:42515
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:35159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:11:47.985581 21589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:11:47.985831 21589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:11:47.985860 21589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:11:47.987035 21589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:11:47.987082 21589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:11:47.987115 21589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:11:47.988628 21589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:35159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.236.20:46663/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:42515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:11:47.988873 21589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:11:47.989133 21589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:11:47.994566 21597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:47.994578 21599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:47.994953 21596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:47.995178 21589 server_base.cc:1034] running on GCE node
I20260929 19:11:47.995365 21589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:11:47.995592 21589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:11:47.996762 21589 hybrid_clock.cc:648] HybridClock initialized: now 1790709107996737 us; error 42 us; skew 500 ppm
Sep 29 19:11:47 dist-test-slave-wftq krb5kdc[20895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709107, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:11:47.999454 21589 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:11:47.999955 21589 webserver.cc:460] Webserver started at http://127.9.236.2:34675/ using document root <none> and password file <none>
I20260929 19:11:48.000197 21589 fs_manager.cc:359] Metadata directory not provided
I20260929 19:11:48.000247 21589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:11:48.000387 21589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 19:11:47 dist-test-slave-wftq krb5kdc[20895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709107, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for HTTP/127.9.236.20@KRBTEST.COM
I20260929 19:11:48.016422 21589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "c6326a69c20a4f9ba7387f3e55de9745"
format_stamp: "Formatted at 2026-09-29 19:11:48 on dist-test-slave-wftq"
server_key: "0bd7f78e3ddefa67cfcefe85e0734a75"
server_key_iv: "c720d04be0316404f7893afc51eff395"
server_key_version: "kuduclusterkey@0"
I20260929 19:11:48.016865 21589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "c6326a69c20a4f9ba7387f3e55de9745"
format_stamp: "Formatted at 2026-09-29 19:11:48 on dist-test-slave-wftq"
server_key: "0bd7f78e3ddefa67cfcefe85e0734a75"
server_key_iv: "c720d04be0316404f7893afc51eff395"
server_key_version: "kuduclusterkey@0"
I20260929 19:11:48.024667 21589 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.000s
Sep 29 19:11:48 dist-test-slave-wftq krb5kdc[20895](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790709108, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:11:48.040321 21606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:48.040639 21589 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:11:48.040823 21589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "c6326a69c20a4f9ba7387f3e55de9745"
format_stamp: "Formatted at 2026-09-29 19:11:48 on dist-test-slave-wftq"
server_key: "0bd7f78e3ddefa67cfcefe85e0734a75"
server_key_iv: "c720d04be0316404f7893afc51eff395"
server_key_version: "kuduclusterkey@0"
I20260929 19:11:48.041039 21589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:11:48.061041 21589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:11:48.078452 21589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:11:48.078620 21589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:11:48.078841 21589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:11:48.079166 21589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:11:48.079193 21589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:48.079216 21589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:11:48.079226 21589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:48.086246 21589 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:44367
I20260929 19:11:48.086663 21589 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:11:48.087559 21719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:44367 every 8 connection(s)
Sep 29 19:11:48 dist-test-slave-wftq krb5kdc[20895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709107, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:11:48.096714 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 21589
I20260929 19:11:48.096837 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:11:48.097051 10160 external_mini_cluster.cc:1428] Setting key 21fddda417f4d04de5e4d4afca59605f
I20260929 19:11:48.098275 21585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:48.087719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:37493 (local address 127.9.236.62:42515)
0929 19:11:48.087780 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:48.087782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:48.088022 (+ 240us) server_negotiation.cc:407] Connection header received
0929 19:11:48.088675 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:48.088677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:48.088688 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:48.088714 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:48.089512 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.090284 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.091706 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.092013 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.092053 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:48.094826 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:11:48.094828 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:11:48.094828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:11:48.094835 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:11:48.096460 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:11:48.097034 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:11:48.097036 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:11:48.097037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:11:48.097160 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:11:48.097441 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:11:48.097443 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:11:48.097443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:11:48.097598 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:11:48.097640 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:48.098045 (+ 405us) server_negotiation.cc:299] Negotiation successful
0929 19:11:48.098157 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 19:11:48.098889 21722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:48.087566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:42515 (local address 127.9.236.2:37493)
0929 19:11:48.087861 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:48.087911 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:48.088561 (+ 650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:48.088882 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:48.088886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:48.089147 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:11:48.089408 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:48.089414 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.090392 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.090393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:48.091611 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:48.091614 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.092139 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.092140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:48.092271 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:48.092928 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:11:48.092938 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:11:48.094699 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:11:48.096557 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:11:48.096565 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:11:48.096569 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:11:48.096927 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:11:48.097233 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:11:48.097234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:11:48.097235 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:11:48.097374 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:11:48.097691 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:11:48.097697 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:11:48.097898 (+ 201us) client_negotiation.cc:769] Sending connection context
0929 19:11:48.098426 (+ 528us) client_negotiation.cc:240] Negotiation successful
0929 19:11:48.098610 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":141,"threads_started":1}
I20260929 19:11:48.101272 21720 heartbeater.cc:347] Connected to a master server at 127.9.236.62:42515
I20260929 19:11:48.101387 21720 heartbeater.cc:464] Registering TS with master...
I20260929 19:11:48.101601 21720 heartbeater.cc:511] Master 127.9.236.62:42515 requested a full tablet report, sending...
I20260929 19:11:48.102159 21371 ts_manager.cc:194] Registered new tserver with Master: c6326a69c20a4f9ba7387f3e55de9745 (127.9.236.2:44367)
I20260929 19:11:48.102718 21371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:37493
WARNING: no policy specified for kudu/127.9.236.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
I20260929 19:11:48.130265 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.236.20:46663/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:42515
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:35159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:11:48.252842 21726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:11:48.253274 21726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:11:48.253378 21726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:11:48.255354 21726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:11:48.255465 21726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:11:48.255548 21726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:11:48.257875 21726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:35159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.236.20:46663/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:42515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:11:48.258395 21726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:11:48.258742 21726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:11:48.267419 21732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:48.267839 21731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:11:48.268082 21734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:11:48.269306 21726 server_base.cc:1034] running on GCE node
I20260929 19:11:48.269552 21726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:11:48.269837 21726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:11:48.285288 21726 hybrid_clock.cc:648] HybridClock initialized: now 1790709108285233 us; error 72 us; skew 500 ppm
Sep 29 19:11:48 dist-test-slave-wftq krb5kdc[20895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709108, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:11:48.293499 21726 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:11:48.294225 21726 webserver.cc:460] Webserver started at http://127.9.236.3:35485/ using document root <none> and password file <none>
I20260929 19:11:48.294574 21726 fs_manager.cc:359] Metadata directory not provided
I20260929 19:11:48.294687 21726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:11:48.294872 21726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 19:11:48 dist-test-slave-wftq krb5kdc[20895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709108, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for HTTP/127.9.236.20@KRBTEST.COM
I20260929 19:11:48.323000 21726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "3701f41f7a3a440ba652d6bd1076f867"
format_stamp: "Formatted at 2026-09-29 19:11:48 on dist-test-slave-wftq"
server_key: "91cdfac344ea77c587b0f3aeab936df3"
server_key_iv: "1acde35d15be767d399c5e13c47c63ce"
server_key_version: "kuduclusterkey@0"
I20260929 19:11:48.323772 21726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "3701f41f7a3a440ba652d6bd1076f867"
format_stamp: "Formatted at 2026-09-29 19:11:48 on dist-test-slave-wftq"
server_key: "91cdfac344ea77c587b0f3aeab936df3"
server_key_iv: "1acde35d15be767d399c5e13c47c63ce"
server_key_version: "kuduclusterkey@0"
I20260929 19:11:48.331498 21726 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.000s
I20260929 19:11:48.349493 21741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:48.352903 21726 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260929 19:11:48.353122 21726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "3701f41f7a3a440ba652d6bd1076f867"
format_stamp: "Formatted at 2026-09-29 19:11:48 on dist-test-slave-wftq"
server_key: "91cdfac344ea77c587b0f3aeab936df3"
server_key_iv: "1acde35d15be767d399c5e13c47c63ce"
server_key_version: "kuduclusterkey@0"
I20260929 19:11:48.353269 21726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:11:48.380885 21726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:11:48.383697 21726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:11:48.383934 21726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:11:48.384534 21726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:11:48.385138 21726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:11:48.385241 21726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:48.385315 21726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:11:48.385373 21726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:11:48.393282 21726 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:35823
I20260929 19:11:48.393826 21726 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:11:48.394595 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 21726
I20260929 19:11:48.394711 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:11:48.394927 10160 external_mini_cluster.cc:1428] Setting key bbe7d0e96ec05defad9ad98481b947d9
I20260929 19:11:48.402504 21854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:35823 every 8 connection(s)
Sep 29 19:11:48 dist-test-slave-wftq krb5kdc[20895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709108, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:11:48.415099 21585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:48.403515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:54925 (local address 127.9.236.62:42515)
0929 19:11:48.403576 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:48.403579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:48.403991 (+ 412us) server_negotiation.cc:407] Connection header received
0929 19:11:48.404801 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:48.404802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:48.404816 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:48.404853 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:48.405865 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.406709 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.408297 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.408610 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.408654 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:48.411637 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:11:48.411639 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:11:48.411639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:11:48.411648 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:11:48.413378 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:11:48.413897 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:11:48.413898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:11:48.413899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:11:48.413959 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:11:48.414262 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:11:48.414263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:11:48.414263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:11:48.414412 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:11:48.414461 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:48.414904 (+ 443us) server_negotiation.cc:299] Negotiation successful
0929 19:11:48.414991 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 19:11:48.416096 21857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:48.403503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:42515 (local address 127.9.236.3:54925)
0929 19:11:48.403844 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:48.403893 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:48.404670 (+ 777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:48.405089 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:48.405094 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:48.405429 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:11:48.405753 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:48.405760 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.406831 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.406832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:48.408191 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:48.408195 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.408756 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.408757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:48.408915 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:48.409664 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:11:48.409676 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:11:48.411518 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:11:48.413475 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:11:48.413482 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:11:48.413485 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:11:48.413806 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:11:48.414023 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:11:48.414024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:11:48.414024 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:11:48.414181 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:11:48.414508 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:11:48.414515 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:11:48.414729 (+ 214us) client_negotiation.cc:769] Sending connection context
0929 19:11:48.415630 (+ 901us) client_negotiation.cc:240] Negotiation successful
0929 19:11:48.415824 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":130,"threads_started":1}
I20260929 19:11:48.416997 21855 heartbeater.cc:347] Connected to a master server at 127.9.236.62:42515
I20260929 19:11:48.417130 21855 heartbeater.cc:464] Registering TS with master...
I20260929 19:11:48.417361 21855 heartbeater.cc:511] Master 127.9.236.62:42515 requested a full tablet report, sending...
I20260929 19:11:48.417840 21371 ts_manager.cc:194] Registered new tserver with Master: 3701f41f7a3a440ba652d6bd1076f867 (127.9.236.3:35823)
I20260929 19:11:48.418437 21371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:54925
I20260929 19:11:48.428243 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:11:48.438594 21585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:48.430496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42378 (local address 127.9.236.62:42515)
0929 19:11:48.430549 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:48.430552 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:48.430959 (+ 407us) server_negotiation.cc:407] Connection header received
0929 19:11:48.431061 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:48.431062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:48.431074 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:48.431101 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:11:48.431873 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.432670 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.433837 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.434155 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.434193 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:48.435662 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:11:48.435664 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:11:48.435664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:11:48.435671 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:11:48.437442 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:11:48.437785 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:11:48.437786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:11:48.437787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:11:48.437835 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:11:48.438024 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:11:48.438025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:11:48.438025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:11:48.438154 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:11:48.438225 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:48.438418 (+ 193us) server_negotiation.cc:299] Negotiation successful
0929 19:11:48.438492 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 19:11:48.440478 21371 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:11:48.442806 21371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:11:48.454149 21869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:48.445758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:35823 (local address 127.0.0.1:46568)
0929 19:11:48.447239 (+ 1481us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:48.447277 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:48.447340 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:48.448897 (+ 1557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:48.448898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:48.448899 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:11:48.448953 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:48.448954 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.450161 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.450162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:48.451588 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:48.451592 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.453617 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.453618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:48.453938 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:48.453957 (+ 19us) client_negotiation.cc:769] Sending connection context
0929 19:11:48.454013 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 19:11:48.454062 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1456,"thread_start_us":82,"threads_started":1}
I20260929 19:11:48.455483 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:48.446008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:39773 (local address 127.0.0.1:45014)
0929 19:11:48.447622 (+ 1614us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:48.447630 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:48.447667 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:48.447880 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:48.447882 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:48.447884 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:11:48.447959 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:48.447960 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.451684 (+ 3724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.451684 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:48.452592 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:48.452595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.455248 (+ 2653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.455249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:48.455311 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:48.455315 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:11:48.455366 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 19:11:48.455410 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1598,"thread_start_us":60,"threads_started":1}
I20260929 19:11:48.455835 21870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:48.446292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46568 (local address 127.9.236.3:35823)
0929 19:11:48.447122 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:48.447127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:48.447409 (+ 282us) server_negotiation.cc:407] Connection header received
0929 19:11:48.447454 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:48.447457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:48.447472 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:48.447531 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:11:48.449023 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.450072 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.452711 (+ 2639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.453443 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.453508 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:48.453584 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:48.455693 (+ 2109us) server_negotiation.cc:299] Negotiation successful
0929 19:11:48.455744 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":792,"thread_start_us":97,"threads_started":1}
I20260929 19:11:48.456578 21789 tablet_service.cc:1467] Processing CreateTablet for tablet 47022bd3289a455c83858ca682e3afde (DEFAULT_TABLE table=test-table [id=cdfb62ae5c024d8fa2b556e91e4aac39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:11:48.456990 21789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47022bd3289a455c83858ca682e3afde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:11:48.460776 21868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:48.446730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:44367 (local address 127.0.0.1:49964)
0929 19:11:48.451566 (+ 4836us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:48.451581 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:48.451640 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:48.457404 (+ 5764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:48.457406 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:48.457419 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:11:48.457457 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:48.457458 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.458665 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.458666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:48.459700 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:48.459703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.460584 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.460585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:48.460642 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:48.460646 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:11:48.460673 (+ 27us) client_negotiation.cc:240] Negotiation successful
0929 19:11:48.460711 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4805,"thread_start_us":65,"threads_started":1}
I20260929 19:11:48.460973 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:48.446474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45014 (local address 127.9.236.1:39773)
0929 19:11:48.447181 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:48.447185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:48.447712 (+ 527us) server_negotiation.cc:407] Connection header received
0929 19:11:48.447746 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:48.447748 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:48.447762 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:48.447817 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:11:48.448026 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.451348 (+ 3322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.454338 (+ 2990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.455033 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.455132 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:48.455214 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:48.460837 (+ 5623us) server_negotiation.cc:299] Negotiation successful
0929 19:11:48.460893 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":51,"threads_started":1}
I20260929 19:11:48.461164 21873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:48.446844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49964 (local address 127.9.236.2:44367)
0929 19:11:48.457214 (+ 10370us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:48.457219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:48.457230 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:11:48.457270 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:48.457272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:48.457289 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:48.457527 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:11:48.457599 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.458563 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.459818 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.460502 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.460927 (+ 425us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:48.461004 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:48.461027 (+ 23us) server_negotiation.cc:299] Negotiation successful
0929 19:11:48.461066 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10324,"thread_start_us":71,"threads_started":1}
I20260929 19:11:48.461330 21874 tablet_bootstrap.cc:492] T 47022bd3289a455c83858ca682e3afde P 3701f41f7a3a440ba652d6bd1076f867: Bootstrap starting.
I20260929 19:11:48.461608 21510 tablet_service.cc:1467] Processing CreateTablet for tablet 47022bd3289a455c83858ca682e3afde (DEFAULT_TABLE table=test-table [id=cdfb62ae5c024d8fa2b556e91e4aac39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:11:48.462044 21510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47022bd3289a455c83858ca682e3afde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:11:48.462769 21874 tablet_bootstrap.cc:654] T 47022bd3289a455c83858ca682e3afde P 3701f41f7a3a440ba652d6bd1076f867: Neither blocks nor log segments found. Creating new log.
I20260929 19:11:48.463073 21874 log.cc:824] T 47022bd3289a455c83858ca682e3afde P 3701f41f7a3a440ba652d6bd1076f867: Log is configured to *not* fsync() on all Append() calls
I20260929 19:11:48.463769 21874 tablet_bootstrap.cc:492] T 47022bd3289a455c83858ca682e3afde P 3701f41f7a3a440ba652d6bd1076f867: No bootstrap required, opened a new log
I20260929 19:11:48.463841 21874 ts_tablet_manager.cc:1397] T 47022bd3289a455c83858ca682e3afde P 3701f41f7a3a440ba652d6bd1076f867: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 19:11:48.465729 21874 raft_consensus.cc:348] T 47022bd3289a455c83858ca682e3afde P 3701f41f7a3a440ba652d6bd1076f867 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3701f41f7a3a440ba652d6bd1076f867" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35823 } } peers { permanent_uuid: "62cd101d6f624d478b8fbca5f1736ad6" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 39773 } } peers { permanent_uuid: "c6326a69c20a4f9ba7387f3e55de9745" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 44367 } }
I20260929 19:11:48.465895 21874 raft_consensus.cc:374] T 47022bd3289a455c83858ca682e3afde P 3701f41f7a3a440ba652d6bd1076f867 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:11:48.465813 21650 tablet_service.cc:1467] Processing CreateTablet for tablet 47022bd3289a455c83858ca682e3afde (DEFAULT_TABLE table=test-table [id=cdfb62ae5c024d8fa2b556e91e4aac39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:11:48.465950 21874 raft_consensus.cc:729] T 47022bd3289a455c83858ca682e3afde P 3701f41f7a3a440ba652d6bd1076f867 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3701f41f7a3a440ba652d6bd1076f867, State: Initialized, Role: FOLLOWER
I20260929 19:11:48.466053 21874 consensus_queue.cc:260] T 47022bd3289a455c83858ca682e3afde P 3701f41f7a3a440ba652d6bd1076f867 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3701f41f7a3a440ba652d6bd1076f867" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35823 } } peers { permanent_uuid: "62cd101d6f624d478b8fbca5f1736ad6" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 39773 } } peers { permanent_uuid: "c6326a69c20a4f9ba7387f3e55de9745" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 44367 } }
I20260929 19:11:48.466101 21650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47022bd3289a455c83858ca682e3afde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:11:48.466291 21874 ts_tablet_manager.cc:1428] T 47022bd3289a455c83858ca682e3afde P 3701f41f7a3a440ba652d6bd1076f867: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 19:11:48.466450 21855 heartbeater.cc:503] Master 127.9.236.62:42515 was elected leader, sending a full tablet report...
I20260929 19:11:48.469260 21876 tablet_bootstrap.cc:492] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6: Bootstrap starting.
I20260929 19:11:48.469879 21876 tablet_bootstrap.cc:654] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6: Neither blocks nor log segments found. Creating new log.
I20260929 19:11:48.470139 21876 log.cc:824] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6: Log is configured to *not* fsync() on all Append() calls
I20260929 19:11:48.470474 21878 tablet_bootstrap.cc:492] T 47022bd3289a455c83858ca682e3afde P c6326a69c20a4f9ba7387f3e55de9745: Bootstrap starting.
I20260929 19:11:48.471342 21878 tablet_bootstrap.cc:654] T 47022bd3289a455c83858ca682e3afde P c6326a69c20a4f9ba7387f3e55de9745: Neither blocks nor log segments found. Creating new log.
I20260929 19:11:48.471616 21878 log.cc:824] T 47022bd3289a455c83858ca682e3afde P c6326a69c20a4f9ba7387f3e55de9745: Log is configured to *not* fsync() on all Append() calls
I20260929 19:11:48.472267 21876 tablet_bootstrap.cc:492] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6: No bootstrap required, opened a new log
I20260929 19:11:48.472333 21876 ts_tablet_manager.cc:1397] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260929 19:11:48.474716 21876 raft_consensus.cc:348] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3701f41f7a3a440ba652d6bd1076f867" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35823 } } peers { permanent_uuid: "62cd101d6f624d478b8fbca5f1736ad6" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 39773 } } peers { permanent_uuid: "c6326a69c20a4f9ba7387f3e55de9745" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 44367 } }
I20260929 19:11:48.474879 21876 raft_consensus.cc:374] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:11:48.474906 21876 raft_consensus.cc:729] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62cd101d6f624d478b8fbca5f1736ad6, State: Initialized, Role: FOLLOWER
I20260929 19:11:48.475013 21876 consensus_queue.cc:260] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3701f41f7a3a440ba652d6bd1076f867" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35823 } } peers { permanent_uuid: "62cd101d6f624d478b8fbca5f1736ad6" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 39773 } } peers { permanent_uuid: "c6326a69c20a4f9ba7387f3e55de9745" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 44367 } }
I20260929 19:11:48.475750 21876 ts_tablet_manager.cc:1428] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260929 19:11:48.475894 21582 heartbeater.cc:503] Master 127.9.236.62:42515 was elected leader, sending a full tablet report...
I20260929 19:11:48.476980 21878 tablet_bootstrap.cc:492] T 47022bd3289a455c83858ca682e3afde P c6326a69c20a4f9ba7387f3e55de9745: No bootstrap required, opened a new log
I20260929 19:11:48.477051 21878 ts_tablet_manager.cc:1397] T 47022bd3289a455c83858ca682e3afde P c6326a69c20a4f9ba7387f3e55de9745: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260929 19:11:48.478778 21878 raft_consensus.cc:348] T 47022bd3289a455c83858ca682e3afde P c6326a69c20a4f9ba7387f3e55de9745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3701f41f7a3a440ba652d6bd1076f867" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35823 } } peers { permanent_uuid: "62cd101d6f624d478b8fbca5f1736ad6" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 39773 } } peers { permanent_uuid: "c6326a69c20a4f9ba7387f3e55de9745" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 44367 } }
I20260929 19:11:48.478930 21878 raft_consensus.cc:374] T 47022bd3289a455c83858ca682e3afde P c6326a69c20a4f9ba7387f3e55de9745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:11:48.478969 21878 raft_consensus.cc:729] T 47022bd3289a455c83858ca682e3afde P c6326a69c20a4f9ba7387f3e55de9745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6326a69c20a4f9ba7387f3e55de9745, State: Initialized, Role: FOLLOWER
I20260929 19:11:48.479538 21878 consensus_queue.cc:260] T 47022bd3289a455c83858ca682e3afde P c6326a69c20a4f9ba7387f3e55de9745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3701f41f7a3a440ba652d6bd1076f867" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35823 } } peers { permanent_uuid: "62cd101d6f624d478b8fbca5f1736ad6" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 39773 } } peers { permanent_uuid: "c6326a69c20a4f9ba7387f3e55de9745" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 44367 } }
I20260929 19:11:48.479784 21878 ts_tablet_manager.cc:1428] T 47022bd3289a455c83858ca682e3afde P c6326a69c20a4f9ba7387f3e55de9745: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:11:48.479882 21720 heartbeater.cc:503] Master 127.9.236.62:42515 was elected leader, sending a full tablet report...
I20260929 19:11:48.496945 21881 raft_consensus.cc:482] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:11:48.499363 21881 raft_consensus.cc:504] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3701f41f7a3a440ba652d6bd1076f867" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35823 } } peers { permanent_uuid: "62cd101d6f624d478b8fbca5f1736ad6" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 39773 } } peers { permanent_uuid: "c6326a69c20a4f9ba7387f3e55de9745" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 44367 } }
I20260929 19:11:48.500625 21881 leader_election.cc:288] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3701f41f7a3a440ba652d6bd1076f867 (127.9.236.3:35823), c6326a69c20a4f9ba7387f3e55de9745 (127.9.236.2:44367)
I20260929 19:11:48.510696 21870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:48.500185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:37873 (local address 127.9.236.3:35823)
0929 19:11:48.500229 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:48.500231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:48.502973 (+ 2742us) server_negotiation.cc:407] Connection header received
0929 19:11:48.503068 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:48.503069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:48.503082 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:48.503107 (+ 25us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:11:48.503474 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.504180 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.508954 (+ 4774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.509550 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.509598 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:48.509609 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:48.510570 (+ 961us) server_negotiation.cc:299] Negotiation successful
0929 19:11:48.510616 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:11:48.510977 21884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:48.499987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:35823 (local address 127.9.236.1:37873)
0929 19:11:48.502898 (+ 2911us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:48.502911 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:48.503002 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:48.503171 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:48.503172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:48.503184 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:11:48.503266 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:48.503267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.504297 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.504298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:48.505558 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:48.505562 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.509663 (+ 4101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.509664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:48.509737 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:48.509741 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:11:48.510834 (+ 1093us) client_negotiation.cc:240] Negotiation successful
0929 19:11:48.510884 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2892,"thread_start_us":102,"threads_started":1}
I20260929 19:11:48.511298 21809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47022bd3289a455c83858ca682e3afde" candidate_uuid: "62cd101d6f624d478b8fbca5f1736ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3701f41f7a3a440ba652d6bd1076f867" is_pre_election: true
I20260929 19:11:48.511487 21809 raft_consensus.cc:2454] T 47022bd3289a455c83858ca682e3afde P 3701f41f7a3a440ba652d6bd1076f867 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62cd101d6f624d478b8fbca5f1736ad6 in term 0.
I20260929 19:11:48.511709 21471 leader_election.cc:302] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3701f41f7a3a440ba652d6bd1076f867, 62cd101d6f624d478b8fbca5f1736ad6; no voters:
I20260929 19:11:48.511847 21881 raft_consensus.cc:2781] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:11:48.511893 21881 raft_consensus.cc:482] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:11:48.511911 21881 raft_consensus.cc:3037] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:11:48.512759 21881 raft_consensus.cc:504] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3701f41f7a3a440ba652d6bd1076f867" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35823 } } peers { permanent_uuid: "62cd101d6f624d478b8fbca5f1736ad6" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 39773 } } peers { permanent_uuid: "c6326a69c20a4f9ba7387f3e55de9745" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 44367 } }
I20260929 19:11:48.513010 21809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47022bd3289a455c83858ca682e3afde" candidate_uuid: "62cd101d6f624d478b8fbca5f1736ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3701f41f7a3a440ba652d6bd1076f867"
I20260929 19:11:48.513085 21809 raft_consensus.cc:3037] T 47022bd3289a455c83858ca682e3afde P 3701f41f7a3a440ba652d6bd1076f867 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:11:48.513875 21809 raft_consensus.cc:2454] T 47022bd3289a455c83858ca682e3afde P 3701f41f7a3a440ba652d6bd1076f867 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62cd101d6f624d478b8fbca5f1736ad6 in term 1.
I20260929 19:11:48.514034 21471 leader_election.cc:302] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3701f41f7a3a440ba652d6bd1076f867, 62cd101d6f624d478b8fbca5f1736ad6; no voters:
I20260929 19:11:48.514240 21881 leader_election.cc:288] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 3701f41f7a3a440ba652d6bd1076f867 (127.9.236.3:35823), c6326a69c20a4f9ba7387f3e55de9745 (127.9.236.2:44367)
I20260929 19:11:48.514289 21881 raft_consensus.cc:2781] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:11:48.514326 21881 raft_consensus.cc:686] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6 [term 1 LEADER]: Becoming Leader. State: Replica: 62cd101d6f624d478b8fbca5f1736ad6, State: Running, Role: LEADER
I20260929 19:11:48.514400 21881 consensus_queue.cc:237] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3701f41f7a3a440ba652d6bd1076f867" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35823 } } peers { permanent_uuid: "62cd101d6f624d478b8fbca5f1736ad6" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 39773 } } peers { permanent_uuid: "c6326a69c20a4f9ba7387f3e55de9745" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 44367 } }
I20260929 19:11:48.515218 21371 catalog_manager.cc:5495] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62cd101d6f624d478b8fbca5f1736ad6 (127.9.236.1). New cstate: current_term: 1 leader_uuid: "62cd101d6f624d478b8fbca5f1736ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3701f41f7a3a440ba652d6bd1076f867" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62cd101d6f624d478b8fbca5f1736ad6" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 39773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6326a69c20a4f9ba7387f3e55de9745" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 44367 } health_report { overall_health: UNKNOWN } } }
I20260929 19:11:48.516258 21873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:48.500548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:42939 (local address 127.9.236.2:44367)
0929 19:11:48.500587 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:48.500589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:48.503381 (+ 2792us) server_negotiation.cc:407] Connection header received
0929 19:11:48.505673 (+ 2292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:48.505674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:48.505686 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:48.505713 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:11:48.506796 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.507519 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.509818 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.510308 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.510351 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:48.510360 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:48.516136 (+ 5776us) server_negotiation.cc:299] Negotiation successful
0929 19:11:48.516194 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:11:48.516513 21883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:48.500356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:44367 (local address 127.9.236.1:42939)
0929 19:11:48.503336 (+ 2980us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:48.503344 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 19:11:48.503402 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:11:48.506644 (+ 3242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:11:48.506646 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:11:48.506648 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:11:48.506718 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:48.506719 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.507630 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.507631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:48.508849 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:11:48.508852 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.510423 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.510424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:11:48.510495 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:48.510498 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:11:48.516367 (+ 5869us) client_negotiation.cc:240] Negotiation successful
0929 19:11:48.516426 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2966,"thread_start_us":97,"threads_started":1}
I20260929 19:11:48.516820 21666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47022bd3289a455c83858ca682e3afde" candidate_uuid: "62cd101d6f624d478b8fbca5f1736ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6326a69c20a4f9ba7387f3e55de9745" is_pre_election: true
I20260929 19:11:48.516957 21666 raft_consensus.cc:2454] T 47022bd3289a455c83858ca682e3afde P c6326a69c20a4f9ba7387f3e55de9745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62cd101d6f624d478b8fbca5f1736ad6 in term 0.
I20260929 19:11:48.517061 21665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47022bd3289a455c83858ca682e3afde" candidate_uuid: "62cd101d6f624d478b8fbca5f1736ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6326a69c20a4f9ba7387f3e55de9745"
I20260929 19:11:48.517139 21665 raft_consensus.cc:3037] T 47022bd3289a455c83858ca682e3afde P c6326a69c20a4f9ba7387f3e55de9745 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:11:48.517901 21665 raft_consensus.cc:2454] T 47022bd3289a455c83858ca682e3afde P c6326a69c20a4f9ba7387f3e55de9745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62cd101d6f624d478b8fbca5f1736ad6 in term 1.
I20260929 19:11:48.526345 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:48.523118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45020 (local address 127.9.236.1:39773)
0929 19:11:48.523164 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 19:11:48.523165 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:11:48.523173 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:11:48.523230 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:11:48.523230 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:11:48.523249 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:11:48.523304 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:11:48.523490 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.524223 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.525348 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:11:48.525653 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:11:48.525705 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:11:48.525883 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:11:48.526004 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:11:48.526067 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 19:11:48.526223 (+ 156us) server_negotiation.cc:299] Negotiation successful
0929 19:11:48.526246 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 19:11:48.528206 21665 raft_consensus.cc:1261] T 47022bd3289a455c83858ca682e3afde P c6326a69c20a4f9ba7387f3e55de9745 [term 1 FOLLOWER]: Refusing update from remote peer 62cd101d6f624d478b8fbca5f1736ad6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:11:48.528530 21885 consensus_queue.cc:1035] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6326a69c20a4f9ba7387f3e55de9745" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 44367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:11:48.531080 21809 raft_consensus.cc:1261] T 47022bd3289a455c83858ca682e3afde P 3701f41f7a3a440ba652d6bd1076f867 [term 1 FOLLOWER]: Refusing update from remote peer 62cd101d6f624d478b8fbca5f1736ad6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:11:48.531325 21885 consensus_queue.cc:1035] T 47022bd3289a455c83858ca682e3afde P 62cd101d6f624d478b8fbca5f1736ad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3701f41f7a3a440ba652d6bd1076f867" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:11:48.534511 21888 mvcc.cc:205] Tried to move back new op lower bound from 7334744508528791552 to 7334744508475711488. Current Snapshot: MvccSnapshot[applied={T|T < 7334744508528791552}]
I20260929 19:11:48.534622 21889 mvcc.cc:205] Tried to move back new op lower bound from 7334744508528791552 to 7334744508475711488. Current Snapshot: MvccSnapshot[applied={T|T < 7334744508528791552}]
I20260929 19:11:48.536003 21891 mvcc.cc:205] Tried to move back new op lower bound from 7334744508528791552 to 7334744508475711488. Current Snapshot: MvccSnapshot[applied={T|T < 7334744508528791552}]
I20260929 19:11:48.539096 21371 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:11:48.539196 21371 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:11:48.539929 21371 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a048bddb4fe42298bc2d1a9ed9d7f2d: Sending DeleteTablet for 3 replicas of tablet 47022bd3289a455c83858ca682e3afde
I20260929 19:11:48.540331 21789 tablet_service.cc:1514] Processing DeleteTablet for tablet 47022bd3289a455c83858ca682e3afde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:11:48 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:46568
I20260929 19:11:48.540524 21510 tablet_service.cc:1514] Processing DeleteTablet for tablet 47022bd3289a455c83858ca682e3afde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:11:48 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:45014
I20260929 19:11:48.540856 21650 tablet_service.cc:1514] Processing DeleteTablet for tablet 47022bd3289a455c83858ca682e3afde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:11:48 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:49964
I20260929 19:11:48.541361 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 21452
I20260929 19:11:48.544200 21897 tablet_replica.cc:330] T 47022bd3289a455c83858ca682e3afde P 3701f41f7a3a440ba652d6bd1076f867: stopping tablet replica
I20260929 19:11:48.544317 21897 raft_consensus.cc:2229] T 47022bd3289a455c83858ca682e3afde P 3701f41f7a3a440ba652d6bd1076f867 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:11:48.544431 21897 raft_consensus.cc:2258] T 47022bd3289a455c83858ca682e3afde P 3701f41f7a3a440ba652d6bd1076f867 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:11:48.544878 21897 ts_tablet_manager.cc:1905] T 47022bd3289a455c83858ca682e3afde P 3701f41f7a3a440ba652d6bd1076f867: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:11:48.546849 21899 tablet_replica.cc:330] T 47022bd3289a455c83858ca682e3afde P c6326a69c20a4f9ba7387f3e55de9745: stopping tablet replica
I20260929 19:11:48.547856 21899 raft_consensus.cc:2229] T 47022bd3289a455c83858ca682e3afde P c6326a69c20a4f9ba7387f3e55de9745 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:11:48.547997 21899 raft_consensus.cc:2258] T 47022bd3289a455c83858ca682e3afde P c6326a69c20a4f9ba7387f3e55de9745 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:11:48.548421 21899 ts_tablet_manager.cc:1905] T 47022bd3289a455c83858ca682e3afde P c6326a69c20a4f9ba7387f3e55de9745: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:11:48.549835 21899 ts_tablet_manager.cc:1918] T 47022bd3289a455c83858ca682e3afde P c6326a69c20a4f9ba7387f3e55de9745: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:11:48.549919 21899 log.cc:1196] T 47022bd3289a455c83858ca682e3afde P c6326a69c20a4f9ba7387f3e55de9745: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/47022bd3289a455c83858ca682e3afde
I20260929 19:11:48.550155 21899 ts_tablet_manager.cc:1939] T 47022bd3289a455c83858ca682e3afde P c6326a69c20a4f9ba7387f3e55de9745: Deleting consensus metadata
I20260929 19:11:48.550539 21357 catalog_manager.cc:4841] TS c6326a69c20a4f9ba7387f3e55de9745 (127.9.236.2:44367): tablet 47022bd3289a455c83858ca682e3afde (table test-table [id=cdfb62ae5c024d8fa2b556e91e4aac39]) successfully deleted
I20260929 19:11:48.553207 21897 ts_tablet_manager.cc:1918] T 47022bd3289a455c83858ca682e3afde P 3701f41f7a3a440ba652d6bd1076f867: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:11:48.553288 21897 log.cc:1196] T 47022bd3289a455c83858ca682e3afde P 3701f41f7a3a440ba652d6bd1076f867: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/47022bd3289a455c83858ca682e3afde
I20260929 19:11:48.554142 21897 ts_tablet_manager.cc:1939] T 47022bd3289a455c83858ca682e3afde P 3701f41f7a3a440ba652d6bd1076f867: Deleting consensus metadata
I20260929 19:11:48.554540 21358 catalog_manager.cc:4841] TS 3701f41f7a3a440ba652d6bd1076f867 (127.9.236.3:35823): tablet 47022bd3289a455c83858ca682e3afde (table test-table [id=cdfb62ae5c024d8fa2b556e91e4aac39]) successfully deleted
W20260929 19:11:48.554924 21355 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.236.1:39773 (error 108)
I20260929 19:11:48.562379 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 21589
I20260929 19:11:48.568681 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 21726
I20260929 19:11:48.570753 21868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:11:48.570619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:39773 (local address 127.0.0.1:45032)
0929 19:11:48.570672 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0929 19:11:48.570714 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.1:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260929 19:11:48.570904 21355 catalog_manager.cc:4584] TS 62cd101d6f624d478b8fbca5f1736ad6 (127.9.236.1:39773): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 47022bd3289a455c83858ca682e3afde: Network error: Client connection negotiation failed: client connection to 127.9.236.1:39773: connect: Connection refused (error 111)
I20260929 19:11:48.575982 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 21339
I20260929 19:11:48.583041 10160 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 19:11:48.917589 10160 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 19:11:48.917701 10160 mini_ranger.cc:66] Stopping Ranger...
I20260929 19:11:49.252733 10160 mini_ranger.cc:68] Stopped Ranger
2026-09-29 19:11:49.252 UTC [20925] LOG: received smart shutdown request
2026-09-29 19:11:49.256 UTC [20925] LOG: background worker "logical replication launcher" (PID 20934) exited with exit code 1
2026-09-29 19:11:49.256 UTC [20929] LOG: shutting down
2026-09-29 19:11:49.419 UTC [20925] LOG: database system is shut down
2026-09-29T19:11:49Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45814 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:11:49 dist-test-slave-wftq krb5kdc[21909](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:11:49 dist-test-slave-wftq krb5kdc[21909](info): set up 2 sockets
Sep 29 19:11:49 dist-test-slave-wftq krb5kdc[21909](info): commencing operation
krb5kdc: starting...
W20260929 19:11:50.469094 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:11:50 dist-test-slave-wftq krb5kdc[21909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:11:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:11:50Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.236.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.236.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.236.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/rangeradmin_127.9.236.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.236.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/rangeradmin_127.9.236.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.236.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.236.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.236.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/rangerlookup_127.9.236.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.236.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/rangerlookup_127.9.236.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.236.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.236.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.236.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/HTTP_127.9.236.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.236.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/HTTP_127.9.236.20@KRBTEST.COM.keytab.
I20260929 19:11:50.637678 10160 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task0UA8iT/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/postgres -l logfile start
2026-09-29 19:11:53.109 UTC [21939] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 19:11:53.110 UTC [21939] LOG: listening on IPv4 address "127.9.236.20", port 33839
2026-09-29 19:11:53.113 UTC [21939] LOG: listening on Unix socket "/tmp/.s.PGSQL.33839"
2026-09-29 19:11:53.132 UTC [21942] LOG: database system was shut down at 2026-09-29 19:11:51 UTC
2026-09-29 19:11:53.136 UTC [21939] LOG: database system is ready to accept connections
I20260929 19:11:54.005322 10160 mini_postgres.cc:96] Postgres bound to 33839
2026-09-29 19:11:54.011 UTC [21950] FATAL: database "slave" does not exist
127.9.236.20:33839 - accepting connections
I20260929 19:11:54.012401 10160 mini_ranger.cc:161] Starting Ranger...
I20260929 19:11:54.028581 10160 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 19:11:54.511283 10160 mini_ranger.cc:87] Created ranger Postgres database
I20260929 19:11:54.511356 10160 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ranger-admin
2026-09-29 19:11:54,785 [I] DB FLAVOR :POSTGRES
2026-09-29 19:11:54,786 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 19:11:54,786 [I] Checking connection..
2026-09-29 19:11:54,786 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 19:11:55,068 [I] Checking connection passed.
2026-09-29 19:11:55,068 [I] --------- Verifying version history table ---------
2026-09-29 19:11:55,069 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 19:11:55,334 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 19:11:55,335 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 19:11:55,588 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 19:11:55,588 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 19:11:55,588 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 19:11:55.837 UTC [22014] WARNING: there is no transaction in progress
2026-09-29 19:11:55,843 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 19:11:55,843 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 19:11:56,106 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 19:11:56,106 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 19:11:56,106 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 19:11:56,374 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 19:11:56,640 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-wftq.c.gcp-upstream.internal','N') ;"
2026-09-29 19:11:56,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 19:11:57,143 [I] Table x_portal_user does not exist in database ranger
2026-09-29 19:11:57,143 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 19:11:57,399 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 19:11:57,399 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 19:11:57,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 19:11:57,940 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 19:11:57,941 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 19:11:58.941 UTC [22122] WARNING: there is no transaction in progress
2026-09-29 19:11:58.957 UTC [22122] WARNING: there is no transaction in progress
2026-09-29 19:11:58.975 UTC [22122] WARNING: there is no transaction in progress
2026-09-29 19:11:58.992 UTC [22122] WARNING: there is no transaction in progress
2026-09-29 19:11:59.009 UTC [22122] WARNING: there is no transaction in progress
2026-09-29 19:11:59.028 UTC [22122] WARNING: there is no transaction in progress
2026-09-29 19:11:59.090 UTC [22122] WARNING: there is no transaction in progress
2026-09-29 19:11:59.097 UTC [22122] WARNING: there is no transaction in progress
2026-09-29 19:11:59.113 UTC [22122] WARNING: there is no transaction in progress
2026-09-29 19:11:59.124 UTC [22122] WARNING: there is no transaction in progress
2026-09-29 19:11:59.134 UTC [22122] WARNING: there is no transaction in progress
2026-09-29 19:11:59.149 UTC [22122] WARNING: there is no transaction in progress
2026-09-29 19:11:59.160 UTC [22122] WARNING: there is no transaction in progress
2026-09-29 19:11:59.167 UTC [22122] WARNING: there is no transaction in progress
2026-09-29 19:11:59.175 UTC [22122] WARNING: there is no transaction in progress
2026-09-29 19:11:59.181 UTC [22122] WARNING: there is no transaction in progress
2026-09-29 19:11:59,832 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 19:11:59,833 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 19:12:00,060 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 19:12:00,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 19:12:00,311 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 19:12:00,312 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 19:12:00,575 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 19:12:00,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 19:12:00,846 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 19:12:01,125 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-wftq.c.gcp-upstream.internal';"
2026-09-29 19:12:01,393 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 19:12:01,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 19:12:01,665 [I] DB_PATCHES have already been applied
I20260929 19:12:01.672178 10160 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ranger-admin:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task0UA8iT/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 19:12:01.672264 10160 mini_ranger.cc:193] Using host: 127.9.236.20
I20260929 19:12:01.675647 10160 mini_ranger.cc:227] Ranger admin URL: http://127.9.236.20:51159
Sep 29, 2026 7:12:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 7:12:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 7:12:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 7:12:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 7:12:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 7:12:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 7:12:02 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 19:12:02.707468 10160 mini_ranger.cc:160] Time spent starting Ranger: real 8.695s user 0.002s sys 0.008s
Sep 29, 2026 7:12:24 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 7:12:24 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 29, 2026 7:12:24 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 7:12:24 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 7:12:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 7:12:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 7:12:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 7:12:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 7:12:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 7:12:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 7:12:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 7:12:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 7:12:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 7:12:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 7:12:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 7:12:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 7:12:24 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 7:12:25 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 19:12:25.676587 10160 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.236.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.236.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.236.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/rangerkms_127.9.236.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.236.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/rangerkms_127.9.236.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.236.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.236.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.236.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/HTTP_127.9.236.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.236.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/HTTP_127.9.236.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 19:12:25 dist-test-slave-wftq krb5kdc[21909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709145, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 19:12:25.765151 10160 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 19:12:25.774737 10160 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 19:12:26.641685 10160 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 19:12:26.641768 10160 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ranger-kms
I20260929 19:12:26.641784 10160 mini_ranger_kms.cc:227] Using postgres at 127.9.236.20:33839
2026-09-29 19:12:26,806 [I] DB FLAVOR :POSTGRES
2026-09-29 19:12:26,806 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 19:12:26,806 [I] Checking connection
2026-09-29 19:12:26,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 19:12:27,036 [I] connection success
2026-09-29 19:12:27,036 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 19:12:27,036 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 19:12:27,036 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 19:12:27,276 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 19:12:27,277 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 19:12:27,277 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 19:12:27,277 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.236.20:33839/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 19:12:27,559 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 19:12:28.104671 10160 mini_ranger_kms.cc:310] Created kms service
I20260929 19:12:28.316530 10160 mini_ranger_kms.cc:326] Created kudu user
I20260929 19:12:28.368628 10160 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 19:12:28 dist-test-slave-wftq krb5kdc[21909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709145, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.236.20@KRBTEST.COM
I20260929 19:12:28.857043 10160 mini_ranger_kms.cc:384] Added ranger policy
I20260929 19:12:28.857203 10160 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ranger-kms:/tmp/dist-test-task0UA8iT/build/release/bin/postgresql.jar:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task0UA8iT/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 19:12:28.857221 10160 mini_ranger_kms.cc:240] Using host: 127.9.236.20
I20260929 19:12:28.860863 10160 mini_ranger_kms.cc:276] Ranger KMS PID: 22303
I20260929 19:12:28.860975 10160 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.236.20:34747
Sep 29, 2026 7:12:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 7:12:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 7:12:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 7:12:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 7:12:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 7:12:30 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 19:12:30.911371 10160 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.146s user 0.008s sys 0.141s
I20260929 19:12:30.911473 10160 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 19:12:30.911525 10160 mini_ranger_kms.cc:401] 127.9.236.20:34747/kms/v1/keys
Sep 29 19:12:31 dist-test-slave-wftq krb5kdc[21909](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790709151, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.236.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 19:12:31 dist-test-slave-wftq krb5kdc[21909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709151, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.236.20@KRBTEST.COM for HTTP/127.9.236.20@KRBTEST.COM
Sep 29 19:12:32 dist-test-slave-wftq krb5kdc[21909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709151, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.236.20@KRBTEST.COM for HTTP/127.9.236.20@KRBTEST.COM
Sep 29 19:12:34 dist-test-slave-wftq krb5kdc[21909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:12:34.557236 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:40099
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:42347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:40099
--ranger_config_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.236.20:34747/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:12:34.643567 22353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:34.643797 22353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:34.643817 22353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:34.644974 22353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:12:34.645015 22353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:34.645027 22353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:34.645038 22353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:12:34.645048 22353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:12:34.646634 22353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:42347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.236.20:34747/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:40099
--ranger_config_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:40099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:34.646840 22353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:34.647032 22353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:34.652647 22359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:34.652758 22353 server_base.cc:1034] running on GCE node
W20260929 19:12:34.652647 22358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:34.652725 22361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:34.653283 22353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:34.653595 22353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:34.654765 22353 hybrid_clock.cc:648] HybridClock initialized: now 1790709154654753 us; error 44 us; skew 500 ppm
Sep 29 19:12:34 dist-test-slave-wftq krb5kdc[21909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709154, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:34.657411 22353 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:12:34.657899 22353 webserver.cc:460] Webserver started at http://127.9.236.62:44131/ using document root <none> and password file <none>
I20260929 19:12:34.658102 22353 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:34.658154 22353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:12:34.658269 22353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 19:12:34 dist-test-slave-wftq krb5kdc[21909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709154, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for HTTP/127.9.236.20@KRBTEST.COM
I20260929 19:12:34.708947 22353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "0dc377c4db39407eaaebd3cb3e841001"
format_stamp: "Formatted at 2026-09-29 19:12:34 on dist-test-slave-wftq"
server_key: "22326c33cafedca63fb5158488bc109d"
server_key_iv: "bc7fcf0438a655d835c2c6d0f6182d04"
server_key_version: "kuduclusterkey@0"
I20260929 19:12:34.709478 22353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "0dc377c4db39407eaaebd3cb3e841001"
format_stamp: "Formatted at 2026-09-29 19:12:34 on dist-test-slave-wftq"
server_key: "22326c33cafedca63fb5158488bc109d"
server_key_iv: "bc7fcf0438a655d835c2c6d0f6182d04"
server_key_version: "kuduclusterkey@0"
I20260929 19:12:34.711385 22353 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260929 19:12:34.730301 22368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:34.730652 22353 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:34.730801 22353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "0dc377c4db39407eaaebd3cb3e841001"
format_stamp: "Formatted at 2026-09-29 19:12:34 on dist-test-slave-wftq"
server_key: "22326c33cafedca63fb5158488bc109d"
server_key_iv: "bc7fcf0438a655d835c2c6d0f6182d04"
server_key_version: "kuduclusterkey@0"
I20260929 19:12:34.730926 22353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:34.756984 22353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:34.758919 22353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:34.759043 22353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:34.763348 22353 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:40099
I20260929 19:12:34.763458 22420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:40099 every 8 connection(s)
I20260929 19:12:34.763774 22353 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:12:34.764462 22421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:12:34.766650 22421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001: Bootstrap starting.
I20260929 19:12:34.767349 22421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001: Neither blocks nor log segments found. Creating new log.
I20260929 19:12:34.767688 22421 log.cc:824] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001: Log is configured to *not* fsync() on all Append() calls
I20260929 19:12:34.768646 22421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001: No bootstrap required, opened a new log
I20260929 19:12:34.771636 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 22353
I20260929 19:12:34.771719 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:12:34.771899 10160 external_mini_cluster.cc:1428] Setting key 08184619e0d4f68c159f3faea2963ab7
I20260929 19:12:34.771909 22421 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc377c4db39407eaaebd3cb3e841001" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40099 } }
I20260929 19:12:34.772161 22421 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:12:34.772364 22421 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc377c4db39407eaaebd3cb3e841001, State: Initialized, Role: FOLLOWER
I20260929 19:12:34.772639 22421 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc377c4db39407eaaebd3cb3e841001" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40099 } }
I20260929 19:12:34.772754 22421 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:12:34.772789 22421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:12:34.772871 22421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:12:34.773897 22421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc377c4db39407eaaebd3cb3e841001" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40099 } }
I20260929 19:12:34.774029 22421 leader_election.cc:302] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0dc377c4db39407eaaebd3cb3e841001; no voters:
I20260929 19:12:34.774312 22421 leader_election.cc:288] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:12:34.774374 22426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:12:34.774608 22426 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [term 1 LEADER]: Becoming Leader. State: Replica: 0dc377c4db39407eaaebd3cb3e841001, State: Running, Role: LEADER
I20260929 19:12:34.774719 22426 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc377c4db39407eaaebd3cb3e841001" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40099 } }
I20260929 19:12:34.775015 22421 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:12:34.775125 22426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0dc377c4db39407eaaebd3cb3e841001" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc377c4db39407eaaebd3cb3e841001" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40099 } } }
I20260929 19:12:34.775249 22426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [sys.catalog]: This master's current role is: LEADER
I20260929 19:12:34.775424 22426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dc377c4db39407eaaebd3cb3e841001. Latest consensus state: current_term: 1 leader_uuid: "0dc377c4db39407eaaebd3cb3e841001" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc377c4db39407eaaebd3cb3e841001" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40099 } } }
I20260929 19:12:34.775508 22426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [sys.catalog]: This master's current role is: LEADER
I20260929 19:12:34.776853 22421 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 19:12:34.777369 22430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 29 19:12:34 dist-test-slave-wftq krb5kdc[21909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:12:34.779989 22430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:12:34.783578 22430 catalog_manager.cc:1347] Generated new cluster ID: 22511dd860d348878846807f8ed793a4
I20260929 19:12:34.783649 22430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:12:34.784276 22425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:34.772453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37960 (local address 127.9.236.62:40099)
0929 19:12:34.772629 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:34.772636 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:34.772658 (+ 22us) server_negotiation.cc:407] Connection header received
0929 19:12:34.772766 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:34.772774 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:34.772836 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:34.772965 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:34.773664 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:34.774949 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:34.775932 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:34.776340 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:34.776431 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:34.779321 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:34.779383 (+ 62us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:34.779383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:34.779480 (+ 97us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:34.782484 (+ 3004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:34.782926 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:34.782929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:34.782930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:34.783027 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:34.783268 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:34.783268 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:34.783269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:34.783598 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:34.783751 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:34.783913 (+ 162us) server_negotiation.cc:299] Negotiation successful
0929 19:12:34.784072 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":79,"threads_started":1}
I20260929 19:12:34.808167 22430 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:12:34.808969 22430 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:12:34.824041 22430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001: Generated new TSK 0
I20260929 19:12:34.824265 22430 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.236.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
I20260929 19:12:35.476330 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.236.20:34747/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:40099
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:42347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:12:35.595400 22466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:35.595629 22466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:35.595661 22466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:35.596902 22466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:35.596946 22466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:35.596982 22466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:12:35.598680 22466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:42347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.236.20:34747/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:40099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:35.599190 22466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:35.599534 22466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:35.605969 22471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:35.606222 22466 server_base.cc:1034] running on GCE node
W20260929 19:12:35.606259 22474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:35.606561 22472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:35.606868 22466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:35.607118 22466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:35.608300 22466 hybrid_clock.cc:648] HybridClock initialized: now 1790709155608266 us; error 40 us; skew 500 ppm
Sep 29 19:12:35 dist-test-slave-wftq krb5kdc[21909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709155, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:35.611838 22466 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:12:35.612450 22466 webserver.cc:460] Webserver started at http://127.9.236.1:46691/ using document root <none> and password file <none>
I20260929 19:12:35.612800 22466 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:35.612896 22466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:12:35.613073 22466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 19:12:35 dist-test-slave-wftq krb5kdc[21909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709155, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for HTTP/127.9.236.20@KRBTEST.COM
I20260929 19:12:35.629648 22466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "0395fa975b704a6ba7e1767d9d25e116"
format_stamp: "Formatted at 2026-09-29 19:12:35 on dist-test-slave-wftq"
server_key: "841f8f572ac0b77a271758620a029445"
server_key_iv: "125d1fbcde8cee75556da23fbf8f11b3"
server_key_version: "kuduclusterkey@0"
I20260929 19:12:35.630288 22466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "0395fa975b704a6ba7e1767d9d25e116"
format_stamp: "Formatted at 2026-09-29 19:12:35 on dist-test-slave-wftq"
server_key: "841f8f572ac0b77a271758620a029445"
server_key_iv: "125d1fbcde8cee75556da23fbf8f11b3"
server_key_version: "kuduclusterkey@0"
I20260929 19:12:35.636400 22466 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260929 19:12:35.656412 22483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:35.656649 22466 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260929 19:12:35.656739 22466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "0395fa975b704a6ba7e1767d9d25e116"
format_stamp: "Formatted at 2026-09-29 19:12:35 on dist-test-slave-wftq"
server_key: "841f8f572ac0b77a271758620a029445"
server_key_iv: "125d1fbcde8cee75556da23fbf8f11b3"
server_key_version: "kuduclusterkey@0"
I20260929 19:12:35.656834 22466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:35.695034 22466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:35.697602 22466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:35.697813 22466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:35.698124 22466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:12:35.698571 22466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:12:35.698659 22466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:35.698756 22466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:12:35.698823 22466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:35.704895 22466 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:43633
I20260929 19:12:35.705404 22466 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:12:35.706564 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 22466
I20260929 19:12:35.706700 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:12:35.706939 10160 external_mini_cluster.cc:1428] Setting key ae35a57d00ea9d500d3d72482028be6f
I20260929 19:12:35.708253 22596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:43633 every 8 connection(s)
Sep 29 19:12:35 dist-test-slave-wftq krb5kdc[21909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709155, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.236.2@KRBTEST.COM; defaulting to no policy
I20260929 19:12:35.726549 22602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:35.713905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:40099 (local address 127.9.236.1:54143)
0929 19:12:35.715215 (+ 1310us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:35.715260 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:35.715829 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:35.716463 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:35.716469 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:35.716748 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:35.717083 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:35.717090 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:35.718221 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:35.718222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:35.719538 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:35.719542 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:35.720052 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:35.720053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:35.720233 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:35.720957 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:35.720971 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:35.722747 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:35.724825 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:35.724832 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:35.724835 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:35.725162 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:35.725407 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:35.725408 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:35.725408 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:35.725540 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:35.725830 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:35.725836 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:35.726031 (+ 195us) client_negotiation.cc:769] Sending connection context
0929 19:12:35.726124 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 19:12:35.726295 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1257,"thread_start_us":111,"threads_started":1}
I20260929 19:12:35.727309 22601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:35.713905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:54143 (local address 127.9.236.62:40099)
0929 19:12:35.716219 (+ 2314us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:35.716222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:35.716231 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:12:35.716259 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:35.716259 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:35.716272 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:35.716343 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:35.717243 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:35.718088 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:35.719640 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:35.719934 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:35.720521 (+ 587us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:35.722901 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:35.722903 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:35.722903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:35.722913 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:35.724720 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:35.725255 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:35.725256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:35.725257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:35.725326 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:35.725607 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:35.725608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:35.725608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:35.725762 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:35.727064 (+ 1302us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:35.727110 (+ 46us) server_negotiation.cc:299] Negotiation successful
0929 19:12:35.727208 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2282,"thread_start_us":95,"threads_started":1}
I20260929 19:12:35.728036 22597 heartbeater.cc:347] Connected to a master server at 127.9.236.62:40099
I20260929 19:12:35.728132 22597 heartbeater.cc:464] Registering TS with master...
I20260929 19:12:35.728318 22597 heartbeater.cc:511] Master 127.9.236.62:40099 requested a full tablet report, sending...
I20260929 19:12:35.728858 22385 ts_manager.cc:194] Registered new tserver with Master: 0395fa975b704a6ba7e1767d9d25e116 (127.9.236.1:43633)
I20260929 19:12:35.729750 22385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:54143
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
I20260929 19:12:35.748555 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.236.20:34747/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:40099
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:42347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 19:12:35 dist-test-slave-wftq krb5kdc[21909](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790709155, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260929 19:12:35.868780 22605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:35.869067 22605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:35.869132 22605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:35.870736 22605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:35.870798 22605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:35.870836 22605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:12:35.872875 22605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:42347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.236.20:34747/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:40099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:35.873147 22605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:35.873484 22605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:35.879628 22613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:35.882234 22611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:35.882375 22610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:35.882686 22605 server_base.cc:1034] running on GCE node
I20260929 19:12:35.882937 22605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:35.883220 22605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:35.886442 22605 hybrid_clock.cc:648] HybridClock initialized: now 1790709155886425 us; error 31 us; skew 500 ppm
Sep 29 19:12:35 dist-test-slave-wftq krb5kdc[21909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709155, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:35.889081 22605 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:12:35.889611 22605 webserver.cc:460] Webserver started at http://127.9.236.2:37477/ using document root <none> and password file <none>
I20260929 19:12:35.889845 22605 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:35.889894 22605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:12:35.890033 22605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 19:12:35 dist-test-slave-wftq krb5kdc[21909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709155, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for HTTP/127.9.236.20@KRBTEST.COM
I20260929 19:12:35.902926 22605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "4ca275b4eb414b44b520f529220af200"
format_stamp: "Formatted at 2026-09-29 19:12:35 on dist-test-slave-wftq"
server_key: "daab5d112b4c85b87d3cad8a2a9613ce"
server_key_iv: "2c65582a94f9f9633336f27881e3987b"
server_key_version: "kuduclusterkey@0"
I20260929 19:12:35.903532 22605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "4ca275b4eb414b44b520f529220af200"
format_stamp: "Formatted at 2026-09-29 19:12:35 on dist-test-slave-wftq"
server_key: "daab5d112b4c85b87d3cad8a2a9613ce"
server_key_iv: "2c65582a94f9f9633336f27881e3987b"
server_key_version: "kuduclusterkey@0"
I20260929 19:12:35.909655 22605 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.002s
I20260929 19:12:35.925367 22621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:35.925704 22605 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260929 19:12:35.925899 22605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "4ca275b4eb414b44b520f529220af200"
format_stamp: "Formatted at 2026-09-29 19:12:35 on dist-test-slave-wftq"
server_key: "daab5d112b4c85b87d3cad8a2a9613ce"
server_key_iv: "2c65582a94f9f9633336f27881e3987b"
server_key_version: "kuduclusterkey@0"
I20260929 19:12:35.926095 22605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:35.947516 22605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:35.950299 22605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:35.950527 22605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:35.950866 22605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:12:35.951342 22605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:12:35.951438 22605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:35.951524 22605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:12:35.951584 22605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:35.971903 22605 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:39605
I20260929 19:12:35.972553 22605 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:12:35.976697 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 22605
I20260929 19:12:35.976802 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:12:35.977059 10160 external_mini_cluster.cc:1428] Setting key f081773b0166af92571687a000bc39e4
I20260929 19:12:35.982687 22734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:39605 every 8 connection(s)
WARNING: no policy specified for kudu/127.9.236.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.3@KRBTEST.COM" created.
Sep 29 19:12:35 dist-test-slave-wftq krb5kdc[21909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709155, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:12:36.001714 22601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:35.987023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:35221 (local address 127.9.236.62:40099)
0929 19:12:35.987080 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:35.987082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:35.988651 (+ 1569us) server_negotiation.cc:407] Connection header received
0929 19:12:35.989680 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:35.989682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:35.989698 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:35.989731 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:35.992579 (+ 2848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:35.993460 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:35.995100 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:35.995406 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:35.995453 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:35.998283 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:35.998284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:35.998285 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:35.998292 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:35.999961 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:36.000478 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:36.000479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:36.000480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:36.000544 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:36.000841 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:36.000842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:36.000842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:36.000992 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:36.001038 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:36.001499 (+ 461us) server_negotiation.cc:299] Negotiation successful
0929 19:12:36.001604 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 19:12:36.002602 22738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:35.987016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:40099 (local address 127.9.236.2:35221)
0929 19:12:35.988485 (+ 1469us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:35.988539 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:35.989539 (+ 1000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:35.989943 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:35.989948 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:35.990277 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:35.990621 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:35.990628 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:35.993604 (+ 2976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:35.993605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:35.994997 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:35.995001 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:35.995566 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:35.995567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:35.995711 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:35.996379 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:35.996391 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:35.998160 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:36.000059 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:36.000066 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:36.000069 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:36.000384 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:36.000615 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:36.000616 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:36.000616 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:36.000760 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:36.001085 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:36.001092 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:36.001328 (+ 236us) client_negotiation.cc:769] Sending connection context
0929 19:12:36.002133 (+ 805us) client_negotiation.cc:240] Negotiation successful
0929 19:12:36.002335 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1410,"thread_start_us":102,"threads_started":1}
I20260929 19:12:36.003338 22735 heartbeater.cc:347] Connected to a master server at 127.9.236.62:40099
I20260929 19:12:36.003450 22735 heartbeater.cc:464] Registering TS with master...
I20260929 19:12:36.003669 22735 heartbeater.cc:511] Master 127.9.236.62:40099 requested a full tablet report, sending...
I20260929 19:12:36.004104 22385 ts_manager.cc:194] Registered new tserver with Master: 4ca275b4eb414b44b520f529220af200 (127.9.236.2:39605)
I20260929 19:12:36.004663 22385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:35221
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
I20260929 19:12:36.022600 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.236.20:34747/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:40099
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:42347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:12:36.160952 22740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:36.161427 22740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:36.161521 22740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:36.163270 22740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:36.163365 22740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:36.163451 22740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:12:36.165553 22740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:42347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.236.20:34747/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:40099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:36.166015 22740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:36.166339 22740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:36.173738 22748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:36.173872 22745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:36.173740 22746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:36.175447 22740 server_base.cc:1034] running on GCE node
I20260929 19:12:36.175969 22740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:36.176285 22740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:36.177445 22740 hybrid_clock.cc:648] HybridClock initialized: now 1790709156177426 us; error 33 us; skew 500 ppm
Sep 29 19:12:36 dist-test-slave-wftq krb5kdc[21909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709156, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:36.180752 22740 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:12:36.181365 22740 webserver.cc:460] Webserver started at http://127.9.236.3:40195/ using document root <none> and password file <none>
I20260929 19:12:36.181659 22740 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:36.181720 22740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:12:36.181911 22740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 19:12:36 dist-test-slave-wftq krb5kdc[21909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709156, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for HTTP/127.9.236.20@KRBTEST.COM
I20260929 19:12:36.205500 22740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "b675ac64627442549aa6f6b510b87bfb"
format_stamp: "Formatted at 2026-09-29 19:12:36 on dist-test-slave-wftq"
server_key: "4fefb8f3d7988a1d66d4dfef0c59eab5"
server_key_iv: "bc8b4745b0df5da504ddbbb9c9ccba36"
server_key_version: "kuduclusterkey@0"
I20260929 19:12:36.206051 22740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "b675ac64627442549aa6f6b510b87bfb"
format_stamp: "Formatted at 2026-09-29 19:12:36 on dist-test-slave-wftq"
server_key: "4fefb8f3d7988a1d66d4dfef0c59eab5"
server_key_iv: "bc8b4745b0df5da504ddbbb9c9ccba36"
server_key_version: "kuduclusterkey@0"
I20260929 19:12:36.208695 22740 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260929 19:12:36.236596 22755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:36.236836 22740 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260929 19:12:36.236910 22740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "b675ac64627442549aa6f6b510b87bfb"
format_stamp: "Formatted at 2026-09-29 19:12:36 on dist-test-slave-wftq"
server_key: "4fefb8f3d7988a1d66d4dfef0c59eab5"
server_key_iv: "bc8b4745b0df5da504ddbbb9c9ccba36"
server_key_version: "kuduclusterkey@0"
I20260929 19:12:36.236992 22740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:36.264695 22740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:36.267079 22740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:36.267285 22740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:36.267566 22740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:12:36.267961 22740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:12:36.268044 22740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:36.268116 22740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:12:36.268173 22740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:36.276299 22740 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:34779
I20260929 19:12:36.276800 22740 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:12:36.287251 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 22740
I20260929 19:12:36.287357 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:12:36.287549 10160 external_mini_cluster.cc:1428] Setting key 65c592d9fdb2a0374cfef5c52673c09f
I20260929 19:12:36.278316 22868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:34779 every 8 connection(s)
Sep 29 19:12:36 dist-test-slave-wftq krb5kdc[21909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709156, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:12:36.294442 22601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:36.279513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:40013 (local address 127.9.236.62:40099)
0929 19:12:36.279563 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:36.279566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:36.279966 (+ 400us) server_negotiation.cc:407] Connection header received
0929 19:12:36.280590 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:36.280592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:36.280605 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:36.280665 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:36.281371 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:36.281886 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:36.283314 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:36.283553 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:36.283614 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:36.291241 (+ 7627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:36.291243 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:36.291244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:36.291252 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:36.292829 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:36.293325 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:36.293326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:36.293326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:36.293393 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:36.293644 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:36.293645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:36.293645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:36.293772 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:36.293817 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:36.294247 (+ 430us) server_negotiation.cc:299] Negotiation successful
0929 19:12:36.294345 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 19:12:36.295181 22871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:36.279515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:40099 (local address 127.9.236.3:40013)
0929 19:12:36.279853 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:36.279893 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:36.280494 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:36.280742 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:36.280746 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:36.280972 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:36.281279 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:36.281284 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:36.281972 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:36.281973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:36.283240 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:36.283243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:36.288436 (+ 5193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:36.288437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:36.288587 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:36.289277 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:36.289287 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:36.291149 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:36.292918 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:36.292924 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:36.292928 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:36.293242 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:36.293457 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:36.293457 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:36.293458 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:36.293581 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:36.293846 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:36.293852 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:36.294100 (+ 248us) client_negotiation.cc:769] Sending connection context
0929 19:12:36.294782 (+ 682us) client_negotiation.cc:240] Negotiation successful
0929 19:12:36.294947 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":84,"threads_started":1}
I20260929 19:12:36.295827 22869 heartbeater.cc:347] Connected to a master server at 127.9.236.62:40099
I20260929 19:12:36.295920 22869 heartbeater.cc:464] Registering TS with master...
I20260929 19:12:36.296098 22869 heartbeater.cc:511] Master 127.9.236.62:40099 requested a full tablet report, sending...
I20260929 19:12:36.296487 22385 ts_manager.cc:194] Registered new tserver with Master: b675ac64627442549aa6f6b510b87bfb (127.9.236.3:34779)
I20260929 19:12:36.296929 22385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:40013
I20260929 19:12:36.298970 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:12:36.299731 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 22466
I20260929 19:12:36.307168 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 22605
I20260929 19:12:36.313231 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 22740
I20260929 19:12:36.318871 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 22353
I20260929 19:12:36.325680 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:40099
--webserver_interface=127.9.236.62
--webserver_port=44131
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:42347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:40099
--ranger_config_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.236.20:34747/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.236.20:34747/kms with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:12:36.425017 22872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:36.425290 22872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:36.425311 22872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:36.426498 22872 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:12:36.426537 22872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:36.426548 22872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:36.426559 22872 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:12:36.426573 22872 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:12:36.428191 22872 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:42347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.236.20:34747/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:40099
--ranger_config_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:40099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=44131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:36.428416 22872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:36.428630 22872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:36.433624 22878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:36.433662 22880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:36.433674 22877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:36.434000 22872 server_base.cc:1034] running on GCE node
I20260929 19:12:36.434269 22872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:36.434585 22872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:36.435724 22872 hybrid_clock.cc:648] HybridClock initialized: now 1790709156435703 us; error 35 us; skew 500 ppm
Sep 29 19:12:36 dist-test-slave-wftq krb5kdc[21909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709156, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:36.438241 22872 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:12:36.438695 22872 webserver.cc:460] Webserver started at http://127.9.236.62:44131/ using document root <none> and password file <none>
I20260929 19:12:36.438910 22872 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:36.438961 22872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 19:12:36 dist-test-slave-wftq krb5kdc[21909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709156, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for HTTP/127.9.236.20@KRBTEST.COM
I20260929 19:12:36.497463 22872 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:36.498454 22888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:36.498684 22872 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:36.498799 22872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "0dc377c4db39407eaaebd3cb3e841001"
format_stamp: "Formatted at 2026-09-29 19:12:34 on dist-test-slave-wftq"
server_key: "22326c33cafedca63fb5158488bc109d"
server_key_iv: "bc7fcf0438a655d835c2c6d0f6182d04"
server_key_version: "kuduclusterkey@0"
I20260929 19:12:36.499359 22872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:36.515872 22872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:36.517680 22872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:36.517802 22872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:36.522529 22872 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:40099
I20260929 19:12:36.522606 22940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:40099 every 8 connection(s)
I20260929 19:12:36.522972 22872 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:12:36.523499 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 22872
I20260929 19:12:36.523763 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.236.20:34747/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:43633
--local_ip_for_outbound_sockets=127.9.236.1
--tserver_master_addrs=127.9.236.62:40099
--webserver_port=46691
--webserver_interface=127.9.236.1
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:42347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.236.20:34747/kms with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:12:36.524434 22941 sys_catalog.cc:263] Verifying existing consensus state
I20260929 19:12:36.525131 22941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001: Bootstrap starting.
I20260929 19:12:36.533057 22941 log.cc:824] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001: Log is configured to *not* fsync() on all Append() calls
I20260929 19:12:36.535097 22941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:12:36.535354 22941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001: Bootstrap complete.
I20260929 19:12:36.537715 22941 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc377c4db39407eaaebd3cb3e841001" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40099 } }
I20260929 19:12:36.537964 22941 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc377c4db39407eaaebd3cb3e841001, State: Initialized, Role: FOLLOWER
I20260929 19:12:36.538112 22941 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc377c4db39407eaaebd3cb3e841001" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40099 } }
I20260929 19:12:36.538182 22941 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:12:36.538221 22941 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:12:36.538266 22941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [term 1 FOLLOWER]: Advancing to term 2
I20260929 19:12:36.539233 22941 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc377c4db39407eaaebd3cb3e841001" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40099 } }
I20260929 19:12:36.539331 22941 leader_election.cc:302] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0dc377c4db39407eaaebd3cb3e841001; no voters:
I20260929 19:12:36.539496 22941 leader_election.cc:288] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 19:12:36.539544 22945 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 19:12:36.539650 22945 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [term 2 LEADER]: Becoming Leader. State: Replica: 0dc377c4db39407eaaebd3cb3e841001, State: Running, Role: LEADER
I20260929 19:12:36.539789 22941 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:12:36.539909 22945 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc377c4db39407eaaebd3cb3e841001" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40099 } }
I20260929 19:12:36.540253 22946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0dc377c4db39407eaaebd3cb3e841001" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc377c4db39407eaaebd3cb3e841001" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40099 } } }
I20260929 19:12:36.540349 22946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [sys.catalog]: This master's current role is: LEADER
I20260929 19:12:36.540683 22947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dc377c4db39407eaaebd3cb3e841001. Latest consensus state: current_term: 2 leader_uuid: "0dc377c4db39407eaaebd3cb3e841001" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc377c4db39407eaaebd3cb3e841001" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40099 } } }
I20260929 19:12:36.540776 22947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001 [sys.catalog]: This master's current role is: LEADER
I20260929 19:12:36.540899 22953 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:12:36.541504 22953 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:12:36.541676 22953 catalog_manager.cc:1259] Loaded cluster ID: 22511dd860d348878846807f8ed793a4
I20260929 19:12:36.541746 22953 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:12:36.541918 22941 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 19:12:36.543028 22953 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:12:36.543216 22953 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001: Loaded TSK: 0
I20260929 19:12:36.543448 22953 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 19:12:36.638666 22943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:36.639045 22943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:36.639140 22943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:36.641058 22943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:36.641142 22943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:36.641192 22943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:12:36.643414 22943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:42347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.236.20:34747/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:43633
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=46691
--tserver_master_addrs=127.9.236.62:40099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:36.643698 22943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:36.644037 22943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:36.651271 22978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:36.651635 22943 server_base.cc:1034] running on GCE node
W20260929 19:12:36.651918 22981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:36.653177 22979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:36.653635 22943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:36.653985 22943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:36.655196 22943 hybrid_clock.cc:648] HybridClock initialized: now 1790709156655172 us; error 44 us; skew 500 ppm
Sep 29 19:12:36 dist-test-slave-wftq krb5kdc[21909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709156, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:36.659039 22943 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:12:36.659691 22943 webserver.cc:460] Webserver started at http://127.9.236.1:46691/ using document root <none> and password file <none>
I20260929 19:12:36.660073 22943 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:36.660171 22943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 19:12:36 dist-test-slave-wftq krb5kdc[21909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709156, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for HTTP/127.9.236.20@KRBTEST.COM
I20260929 19:12:36.696540 22943 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:12:36.703979 22989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:36.704295 22943 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260929 19:12:36.704396 22943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "0395fa975b704a6ba7e1767d9d25e116"
format_stamp: "Formatted at 2026-09-29 19:12:35 on dist-test-slave-wftq"
server_key: "841f8f572ac0b77a271758620a029445"
server_key_iv: "125d1fbcde8cee75556da23fbf8f11b3"
server_key_version: "kuduclusterkey@0"
I20260929 19:12:36.704979 22943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:36.727296 22943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:36.729770 22943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:36.729988 22943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:36.730289 22943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:12:36.730732 22943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:12:36.730824 22943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:36.730935 22943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:12:36.731012 22943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:36.738255 22943 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:43633
I20260929 19:12:36.738813 22943 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:12:36.739954 23102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:43633 every 8 connection(s)
I20260929 19:12:36.741469 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 22943
I20260929 19:12:36.741788 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.236.20:34747/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:39605
--local_ip_for_outbound_sockets=127.9.236.2
--tserver_master_addrs=127.9.236.62:40099
--webserver_port=37477
--webserver_interface=127.9.236.2
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:42347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.236.20:34747/kms with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 19:12:36 dist-test-slave-wftq krb5kdc[21909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709156, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:12:36.762753 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:36.741057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:54697 (local address 127.9.236.62:40099)
0929 19:12:36.750166 (+ 9109us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:36.750177 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:36.750229 (+ 52us) server_negotiation.cc:407] Connection header received
0929 19:12:36.750581 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:36.750593 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:36.750660 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:36.750790 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:36.751819 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:36.753311 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:36.754877 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:36.755533 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:36.755622 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:36.758785 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:36.758789 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:36.758790 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:36.758802 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:36.760723 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:36.761213 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:36.761217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:36.761217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:36.761330 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:36.761580 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:36.761581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:36.761581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:36.761867 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:36.761932 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:36.762365 (+ 433us) server_negotiation.cc:299] Negotiation successful
0929 19:12:36.762503 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9041,"thread_start_us":2263,"threads_started":1}
I20260929 19:12:36.763888 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:36.740857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:40099 (local address 127.9.236.1:54697)
0929 19:12:36.749423 (+ 8566us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:36.749494 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:36.750292 (+ 798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:36.751038 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:36.751043 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:36.751347 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:36.751669 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:36.751676 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:36.753444 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:36.753445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:36.754771 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:36.754775 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:36.756003 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:36.756003 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:36.756222 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:36.756855 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:36.756866 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:36.758661 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:36.760819 (+ 2158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:36.760825 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:36.760827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:36.761125 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:36.761398 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:36.761398 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:36.761399 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:36.761515 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:36.761967 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:36.761973 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:36.762143 (+ 170us) client_negotiation.cc:769] Sending connection context
0929 19:12:36.763509 (+ 1366us) client_negotiation.cc:240] Negotiation successful
0929 19:12:36.763672 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8502,"thread_start_us":93,"threads_started":1}
I20260929 19:12:36.764878 23103 heartbeater.cc:347] Connected to a master server at 127.9.236.62:40099
I20260929 19:12:36.765026 23103 heartbeater.cc:464] Registering TS with master...
I20260929 19:12:36.765301 23103 heartbeater.cc:511] Master 127.9.236.62:40099 requested a full tablet report, sending...
W20260929 19:12:36.766322 23103 heartbeater.cc:644] Failed to heartbeat to 127.9.236.62:40099 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 19:12:36.766391 23103 heartbeater.cc:464] Registering TS with master...
I20260929 19:12:36.766448 23103 heartbeater.cc:511] Master 127.9.236.62:40099 requested a full tablet report, sending...
I20260929 19:12:36.766855 23103 heartbeater.cc:464] Registering TS with master...
I20260929 19:12:36.766927 23103 heartbeater.cc:511] Master 127.9.236.62:40099 requested a full tablet report, sending...
W20260929 19:12:36.767266 23103 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 19:12:36.853654 23106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:36.853863 23106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:36.853892 23106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:36.855028 23106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:36.855070 23106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:36.855104 23106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:12:36.856535 23106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:42347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.236.20:34747/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:39605
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=37477
--tserver_master_addrs=127.9.236.62:40099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:36.856779 23106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:36.857003 23106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:36.862172 23115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:36.862246 23106 server_base.cc:1034] running on GCE node
W20260929 19:12:36.862375 23113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:36.862531 23112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:36.862747 23106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:36.862979 23106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:36.864112 23106 hybrid_clock.cc:648] HybridClock initialized: now 1790709156864102 us; error 26 us; skew 500 ppm
Sep 29 19:12:36 dist-test-slave-wftq krb5kdc[21909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709156, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:36.866588 23106 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:12:36.867029 23106 webserver.cc:460] Webserver started at http://127.9.236.2:37477/ using document root <none> and password file <none>
I20260929 19:12:36.867251 23106 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:36.867318 23106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 19:12:36 dist-test-slave-wftq krb5kdc[21909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709156, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for HTTP/127.9.236.20@KRBTEST.COM
I20260929 19:12:36.881701 23106 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:36.882383 23123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:36.882530 23106 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:36.882614 23106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "4ca275b4eb414b44b520f529220af200"
format_stamp: "Formatted at 2026-09-29 19:12:35 on dist-test-slave-wftq"
server_key: "daab5d112b4c85b87d3cad8a2a9613ce"
server_key_iv: "2c65582a94f9f9633336f27881e3987b"
server_key_version: "kuduclusterkey@0"
I20260929 19:12:36.882942 23106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:36.903119 23106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:36.905718 23106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:36.905864 23106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:36.906077 23106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:12:36.906414 23106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:12:36.906451 23106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:36.906481 23106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:12:36.906497 23106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:36.913054 23106 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:39605
I20260929 19:12:36.913226 23236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:39605 every 8 connection(s)
I20260929 19:12:36.913506 23106 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:12:36.918478 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 23106
I20260929 19:12:36.918838 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.236.20:34747/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:34779
--local_ip_for_outbound_sockets=127.9.236.3
--tserver_master_addrs=127.9.236.62:40099
--webserver_port=40195
--webserver_interface=127.9.236.3
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:42347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.236.20:34747/kms with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 19:12:36 dist-test-slave-wftq krb5kdc[21909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709156, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:12:36.924185 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:36.914348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:40139 (local address 127.9.236.62:40099)
0929 19:12:36.914397 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:36.914399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:36.914717 (+ 318us) server_negotiation.cc:407] Connection header received
0929 19:12:36.915330 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:36.915332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:36.915349 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:36.915429 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:36.916143 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:36.916981 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:36.918541 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:36.918820 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:36.918881 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:36.921305 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:36.921319 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:36.921319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:36.921326 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:36.922969 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:36.923327 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:36.923328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:36.923328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:36.923384 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:36.923585 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:36.923586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:36.923586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:36.923702 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:36.923757 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:36.924010 (+ 253us) server_negotiation.cc:299] Negotiation successful
0929 19:12:36.924092 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:12:36.924535 23239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:36.914383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:40099 (local address 127.9.236.2:40139)
0929 19:12:36.914595 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:36.914634 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:36.915169 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:36.915520 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:36.915525 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:36.915751 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:36.916027 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:36.916033 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:36.917092 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:36.917094 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:36.918453 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:36.918456 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:36.919097 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:36.919098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:36.919213 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:36.919794 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:36.919803 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:36.921193 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:36.923058 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:36.923065 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:36.923067 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:36.923257 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:36.923438 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:36.923438 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:36.923439 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:36.923532 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:36.923753 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:36.923758 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:36.923896 (+ 138us) client_negotiation.cc:769] Sending connection context
0929 19:12:36.923982 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 19:12:36.924114 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":120,"threads_started":1}
I20260929 19:12:36.924955 23237 heartbeater.cc:347] Connected to a master server at 127.9.236.62:40099
I20260929 19:12:36.925042 23237 heartbeater.cc:464] Registering TS with master...
I20260929 19:12:36.925256 23237 heartbeater.cc:511] Master 127.9.236.62:40099 requested a full tablet report, sending...
W20260929 19:12:36.926090 23237 heartbeater.cc:644] Failed to heartbeat to 127.9.236.62:40099 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 19:12:36.926198 23237 heartbeater.cc:464] Registering TS with master...
I20260929 19:12:36.926291 23237 heartbeater.cc:511] Master 127.9.236.62:40099 requested a full tablet report, sending...
I20260929 19:12:36.926800 23237 heartbeater.cc:464] Registering TS with master...
I20260929 19:12:36.926882 23237 heartbeater.cc:511] Master 127.9.236.62:40099 requested a full tablet report, sending...
W20260929 19:12:36.927273 23237 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 19:12:37.027066 23240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:37.027415 23240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:37.027462 23240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:37.029453 23240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:37.029521 23240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:37.029563 23240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:12:37.031785 23240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:42347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.236.20:34747/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:34779
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=40195
--tserver_master_addrs=127.9.236.62:40099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:37.032073 23240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:37.032387 23240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:37.038862 23246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:37.038872 23247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:37.038872 23249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:37.039606 23240 server_base.cc:1034] running on GCE node
I20260929 19:12:37.039901 23240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:37.040189 23240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:37.041363 23240 hybrid_clock.cc:648] HybridClock initialized: now 1790709157041349 us; error 33 us; skew 500 ppm
Sep 29 19:12:37 dist-test-slave-wftq krb5kdc[21909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709157, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:37.043939 23240 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:12:37.044425 23240 webserver.cc:460] Webserver started at http://127.9.236.3:40195/ using document root <none> and password file <none>
I20260929 19:12:37.044662 23240 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:37.044719 23240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 19:12:37 dist-test-slave-wftq krb5kdc[21909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709157, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for HTTP/127.9.236.20@KRBTEST.COM
I20260929 19:12:37.064129 23240 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260929 19:12:37.065018 23257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:37.065279 23240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:37.065399 23240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "b675ac64627442549aa6f6b510b87bfb"
format_stamp: "Formatted at 2026-09-29 19:12:36 on dist-test-slave-wftq"
server_key: "4fefb8f3d7988a1d66d4dfef0c59eab5"
server_key_iv: "bc8b4745b0df5da504ddbbb9c9ccba36"
server_key_version: "kuduclusterkey@0"
I20260929 19:12:37.065924 23240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:37.079229 23240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:37.080991 23240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:37.081159 23240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:37.081382 23240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:12:37.081712 23240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:12:37.081748 23240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:37.081777 23240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:12:37.081794 23240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:37.088775 23240 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:34779
I20260929 19:12:37.089279 23240 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:12:37.089737 23370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:34779 every 8 connection(s)
I20260929 19:12:37.097525 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 23240
Sep 29 19:12:37 dist-test-slave-wftq krb5kdc[21909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709157, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:12:37.103649 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:37.094262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:40099 (local address 127.9.236.3:43745)
0929 19:12:37.094496 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:37.094548 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:37.095240 (+ 692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:37.095615 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:37.095620 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:37.095820 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:37.096009 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:37.096013 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:37.096691 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:37.096692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:37.097706 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:37.097709 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:37.098503 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:37.098504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:37.098630 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:37.099279 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:37.099289 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:37.100651 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:37.102211 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:37.102217 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:37.102220 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:37.102490 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:37.102687 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:37.102688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:37.102688 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:37.102817 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:37.103086 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:37.103092 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:37.103255 (+ 163us) client_negotiation.cc:769] Sending connection context
0929 19:12:37.103359 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 19:12:37.103481 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":121,"threads_started":1}
I20260929 19:12:37.104092 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:37.094220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:43745 (local address 127.9.236.62:40099)
0929 19:12:37.094280 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:37.094283 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:37.094665 (+ 382us) server_negotiation.cc:407] Connection header received
0929 19:12:37.095384 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:37.095386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:37.095399 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:37.095430 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:37.096102 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:37.096608 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:37.098122 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:37.098359 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:37.098394 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:37.100734 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:37.100735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:37.100736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:37.100742 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:37.102120 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:37.102571 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:37.102572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:37.102572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:37.102628 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:37.102881 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:37.102882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:37.102882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:37.103007 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:37.103062 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:37.103919 (+ 857us) server_negotiation.cc:299] Negotiation successful
0929 19:12:37.104006 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 19:12:37.104606 23371 heartbeater.cc:347] Connected to a master server at 127.9.236.62:40099
I20260929 19:12:37.104712 23371 heartbeater.cc:464] Registering TS with master...
I20260929 19:12:37.104924 23371 heartbeater.cc:511] Master 127.9.236.62:40099 requested a full tablet report, sending...
W20260929 19:12:37.105775 23371 heartbeater.cc:644] Failed to heartbeat to 127.9.236.62:40099 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 19:12:37.105840 23371 heartbeater.cc:464] Registering TS with master...
I20260929 19:12:37.105904 23371 heartbeater.cc:511] Master 127.9.236.62:40099 requested a full tablet report, sending...
I20260929 19:12:37.106467 23371 heartbeater.cc:464] Registering TS with master...
I20260929 19:12:37.106554 23371 heartbeater.cc:511] Master 127.9.236.62:40099 requested a full tablet report, sending...
W20260929 19:12:37.107167 23371 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 19:12:37.108628 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:37.097955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37970 (local address 127.9.236.62:40099)
0929 19:12:37.098880 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:37.098882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:37.098888 (+ 6us) server_negotiation.cc:407] Connection header received
0929 19:12:37.098904 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:37.098904 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:37.098913 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:37.098966 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:37.099699 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:37.101976 (+ 2277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:37.103495 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:37.103817 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:37.103897 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:37.105674 (+ 1777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:37.105675 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:37.105675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:37.105683 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:37.107461 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:37.107784 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:37.107785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:37.107785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:37.107853 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:37.108018 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:37.108019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:37.108019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:37.108156 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:37.108369 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:37.108417 (+ 48us) server_negotiation.cc:299] Negotiation successful
0929 19:12:37.108514 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":901,"thread_start_us":104,"threads_started":1}
Sep 29 19:12:37 dist-test-slave-wftq krb5kdc[21909](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790709157, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:37.767477 23103 heartbeater.cc:464] Registering TS with master...
I20260929 19:12:37.767693 23103 heartbeater.cc:511] Master 127.9.236.62:40099 requested a full tablet report, sending...
I20260929 19:12:37.768167 22905 ts_manager.cc:194] Registered new tserver with Master: 0395fa975b704a6ba7e1767d9d25e116 (127.9.236.1:43633)
I20260929 19:12:37.769268 22905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:54697
I20260929 19:12:37.927460 23237 heartbeater.cc:464] Registering TS with master...
I20260929 19:12:37.927619 23237 heartbeater.cc:511] Master 127.9.236.62:40099 requested a full tablet report, sending...
I20260929 19:12:37.927924 22905 ts_manager.cc:194] Registered new tserver with Master: 4ca275b4eb414b44b520f529220af200 (127.9.236.2:39605)
I20260929 19:12:37.928458 22905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:40139
Sep 29 19:12:38 dist-test-slave-wftq krb5kdc[21909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709157, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for HTTP/127.9.236.20@KRBTEST.COM
I20260929 19:12:38.107345 23371 heartbeater.cc:464] Registering TS with master...
I20260929 19:12:38.107508 23371 heartbeater.cc:511] Master 127.9.236.62:40099 requested a full tablet report, sending...
I20260929 19:12:38.108964 22905 ts_manager.cc:194] Registered new tserver with Master: b675ac64627442549aa6f6b510b87bfb (127.9.236.3:34779)
I20260929 19:12:38.109596 22905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:43745
I20260929 19:12:38.117717 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:12:38.127054 23392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:38.119024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37980 (local address 127.9.236.62:40099)
0929 19:12:38.119316 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:38.119318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:38.119495 (+ 177us) server_negotiation.cc:407] Connection header received
0929 19:12:38.119520 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:38.119520 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:38.119535 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:38.119598 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:38.120372 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.121253 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.122292 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.122609 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.122665 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:38.124182 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:38.124184 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:38.124184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:38.124191 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:38.125934 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:38.126265 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:38.126266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:38.126267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:38.126319 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:38.126472 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:38.126473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:38.126473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:38.126587 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:38.126670 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:38.126870 (+ 200us) server_negotiation.cc:299] Negotiation successful
0929 19:12:38.126946 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":81,"threads_started":1}
I20260929 19:12:38.129191 22905 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37980:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:12:38.131131 22905 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:12:38.138924 23398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:38.133982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:43633 (local address 127.0.0.1:35952)
0929 19:12:38.134296 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:38.134305 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:38.134353 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:38.134891 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:38.134892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:38.134893 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:12:38.134923 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:38.134924 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.136208 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.136209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:38.137647 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:38.137650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.138615 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.138616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:38.138699 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:38.138716 (+ 17us) client_negotiation.cc:769] Sending connection context
0929 19:12:38.138763 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 19:12:38.138825 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":13,"thread_start_us":136,"threads_started":1}
I20260929 19:12:38.139357 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:38.134042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49204 (local address 127.9.236.3:34779)
0929 19:12:38.134234 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:38.134239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:38.134476 (+ 237us) server_negotiation.cc:407] Connection header received
0929 19:12:38.134516 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:38.134518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:38.134535 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:38.134650 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:12:38.134907 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.136073 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.137745 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.138774 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.138824 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:38.138908 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:38.139228 (+ 320us) server_negotiation.cc:299] Negotiation successful
0929 19:12:38.139277 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":91,"threads_started":1}
I20260929 19:12:38.139358 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:38.134250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:39605 (local address 127.0.0.1:35926)
0929 19:12:38.135093 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:38.135107 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:38.135172 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:38.135483 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:38.135484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:38.135486 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:12:38.135595 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:38.135599 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.136921 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.136923 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:38.138112 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:38.138115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.139064 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.139065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:38.139136 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:38.139172 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 19:12:38.139200 (+ 28us) client_negotiation.cc:240] Negotiation successful
0929 19:12:38.139255 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"thread_start_us":75,"threads_started":1}
I20260929 19:12:38.139170 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:38.133911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:34779 (local address 127.0.0.1:49204)
0929 19:12:38.134185 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:38.134226 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:38.134310 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:38.134757 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:38.134760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:38.134763 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:12:38.134833 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:38.134834 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.136171 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.136172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:38.137647 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:38.137650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.138962 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.138962 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:38.139014 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:38.139018 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:12:38.139064 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 19:12:38.139104 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":1,"thread_start_us":82,"threads_started":1}
I20260929 19:12:38.139720 23403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:38.134683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35926 (local address 127.9.236.2:39605)
0929 19:12:38.135293 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:38.135299 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:38.135310 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:12:38.135350 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:38.135353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:38.135370 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:38.135670 (+ 300us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:12:38.135739 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.136822 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.138211 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.138902 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.139501 (+ 599us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:38.139585 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:38.139612 (+ 27us) server_negotiation.cc:299] Negotiation successful
0929 19:12:38.139651 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":63,"threads_started":1}
I20260929 19:12:38.140101 23402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:38.134539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35952 (local address 127.9.236.1:43633)
0929 19:12:38.134724 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:38.134729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:38.134740 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:12:38.134774 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:38.134776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:38.134790 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:38.134987 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:12:38.135063 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.136124 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.137750 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.138513 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.139361 (+ 848us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:38.139929 (+ 568us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:38.139956 (+ 27us) server_negotiation.cc:299] Negotiation successful
0929 19:12:38.140004 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":66,"threads_started":1}
I20260929 19:12:38.140275 23287 tablet_service.cc:1467] Processing CreateTablet for tablet 4806f22122474ae1be634e58a4b46cec (DEFAULT_TABLE table=test-table [id=e46168b10f6d4d68a1a591f7c5e06249]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:12:38.140458 23155 tablet_service.cc:1467] Processing CreateTablet for tablet 4806f22122474ae1be634e58a4b46cec (DEFAULT_TABLE table=test-table [id=e46168b10f6d4d68a1a591f7c5e06249]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:12:38.140710 23287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4806f22122474ae1be634e58a4b46cec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:12:38.140853 23155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4806f22122474ae1be634e58a4b46cec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:12:38.140951 23026 tablet_service.cc:1467] Processing CreateTablet for tablet 4806f22122474ae1be634e58a4b46cec (DEFAULT_TABLE table=test-table [id=e46168b10f6d4d68a1a591f7c5e06249]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:12:38.141278 23026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4806f22122474ae1be634e58a4b46cec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:12:38.143836 23404 tablet_bootstrap.cc:492] T 4806f22122474ae1be634e58a4b46cec P b675ac64627442549aa6f6b510b87bfb: Bootstrap starting.
I20260929 19:12:38.144378 23405 tablet_bootstrap.cc:492] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200: Bootstrap starting.
I20260929 19:12:38.144798 23404 tablet_bootstrap.cc:654] T 4806f22122474ae1be634e58a4b46cec P b675ac64627442549aa6f6b510b87bfb: Neither blocks nor log segments found. Creating new log.
I20260929 19:12:38.144845 23406 tablet_bootstrap.cc:492] T 4806f22122474ae1be634e58a4b46cec P 0395fa975b704a6ba7e1767d9d25e116: Bootstrap starting.
I20260929 19:12:38.145303 23405 tablet_bootstrap.cc:654] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200: Neither blocks nor log segments found. Creating new log.
I20260929 19:12:38.145287 23404 log.cc:824] T 4806f22122474ae1be634e58a4b46cec P b675ac64627442549aa6f6b510b87bfb: Log is configured to *not* fsync() on all Append() calls
I20260929 19:12:38.145502 23406 tablet_bootstrap.cc:654] T 4806f22122474ae1be634e58a4b46cec P 0395fa975b704a6ba7e1767d9d25e116: Neither blocks nor log segments found. Creating new log.
I20260929 19:12:38.145628 23405 log.cc:824] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200: Log is configured to *not* fsync() on all Append() calls
I20260929 19:12:38.145807 23406 log.cc:824] T 4806f22122474ae1be634e58a4b46cec P 0395fa975b704a6ba7e1767d9d25e116: Log is configured to *not* fsync() on all Append() calls
I20260929 19:12:38.146286 23404 tablet_bootstrap.cc:492] T 4806f22122474ae1be634e58a4b46cec P b675ac64627442549aa6f6b510b87bfb: No bootstrap required, opened a new log
I20260929 19:12:38.146355 23404 ts_tablet_manager.cc:1397] T 4806f22122474ae1be634e58a4b46cec P b675ac64627442549aa6f6b510b87bfb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260929 19:12:38.146467 23405 tablet_bootstrap.cc:492] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200: No bootstrap required, opened a new log
I20260929 19:12:38.146516 23405 ts_tablet_manager.cc:1397] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:12:38.146572 23406 tablet_bootstrap.cc:492] T 4806f22122474ae1be634e58a4b46cec P 0395fa975b704a6ba7e1767d9d25e116: No bootstrap required, opened a new log
I20260929 19:12:38.146631 23406 ts_tablet_manager.cc:1397] T 4806f22122474ae1be634e58a4b46cec P 0395fa975b704a6ba7e1767d9d25e116: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:12:38.148706 23404 raft_consensus.cc:348] T 4806f22122474ae1be634e58a4b46cec P b675ac64627442549aa6f6b510b87bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0395fa975b704a6ba7e1767d9d25e116" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43633 } } peers { permanent_uuid: "b675ac64627442549aa6f6b510b87bfb" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34779 } } peers { permanent_uuid: "4ca275b4eb414b44b520f529220af200" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39605 } }
I20260929 19:12:38.148749 23406 raft_consensus.cc:348] T 4806f22122474ae1be634e58a4b46cec P 0395fa975b704a6ba7e1767d9d25e116 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0395fa975b704a6ba7e1767d9d25e116" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43633 } } peers { permanent_uuid: "b675ac64627442549aa6f6b510b87bfb" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34779 } } peers { permanent_uuid: "4ca275b4eb414b44b520f529220af200" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39605 } }
I20260929 19:12:38.148898 23404 raft_consensus.cc:374] T 4806f22122474ae1be634e58a4b46cec P b675ac64627442549aa6f6b510b87bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:12:38.148916 23406 raft_consensus.cc:374] T 4806f22122474ae1be634e58a4b46cec P 0395fa975b704a6ba7e1767d9d25e116 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:12:38.148808 23405 raft_consensus.cc:348] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0395fa975b704a6ba7e1767d9d25e116" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43633 } } peers { permanent_uuid: "b675ac64627442549aa6f6b510b87bfb" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34779 } } peers { permanent_uuid: "4ca275b4eb414b44b520f529220af200" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39605 } }
I20260929 19:12:38.148931 23404 raft_consensus.cc:729] T 4806f22122474ae1be634e58a4b46cec P b675ac64627442549aa6f6b510b87bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b675ac64627442549aa6f6b510b87bfb, State: Initialized, Role: FOLLOWER
I20260929 19:12:38.148953 23405 raft_consensus.cc:374] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:12:38.148952 23406 raft_consensus.cc:729] T 4806f22122474ae1be634e58a4b46cec P 0395fa975b704a6ba7e1767d9d25e116 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0395fa975b704a6ba7e1767d9d25e116, State: Initialized, Role: FOLLOWER
I20260929 19:12:38.148984 23405 raft_consensus.cc:729] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ca275b4eb414b44b520f529220af200, State: Initialized, Role: FOLLOWER
I20260929 19:12:38.149055 23404 consensus_queue.cc:260] T 4806f22122474ae1be634e58a4b46cec P b675ac64627442549aa6f6b510b87bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0395fa975b704a6ba7e1767d9d25e116" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43633 } } peers { permanent_uuid: "b675ac64627442549aa6f6b510b87bfb" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34779 } } peers { permanent_uuid: "4ca275b4eb414b44b520f529220af200" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39605 } }
I20260929 19:12:38.149068 23406 consensus_queue.cc:260] T 4806f22122474ae1be634e58a4b46cec P 0395fa975b704a6ba7e1767d9d25e116 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0395fa975b704a6ba7e1767d9d25e116" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43633 } } peers { permanent_uuid: "b675ac64627442549aa6f6b510b87bfb" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34779 } } peers { permanent_uuid: "4ca275b4eb414b44b520f529220af200" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39605 } }
I20260929 19:12:38.149175 23405 consensus_queue.cc:260] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0395fa975b704a6ba7e1767d9d25e116" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43633 } } peers { permanent_uuid: "b675ac64627442549aa6f6b510b87bfb" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34779 } } peers { permanent_uuid: "4ca275b4eb414b44b520f529220af200" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39605 } }
I20260929 19:12:38.149336 23404 ts_tablet_manager.cc:1428] T 4806f22122474ae1be634e58a4b46cec P b675ac64627442549aa6f6b510b87bfb: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260929 19:12:38.149348 23406 ts_tablet_manager.cc:1428] T 4806f22122474ae1be634e58a4b46cec P 0395fa975b704a6ba7e1767d9d25e116: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:12:38.149407 23405 ts_tablet_manager.cc:1428] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:12:38.149513 23237 heartbeater.cc:503] Master 127.9.236.62:40099 was elected leader, sending a full tablet report...
I20260929 19:12:38.149509 23371 heartbeater.cc:503] Master 127.9.236.62:40099 was elected leader, sending a full tablet report...
I20260929 19:12:38.149940 23103 heartbeater.cc:503] Master 127.9.236.62:40099 was elected leader, sending a full tablet report...
W20260929 19:12:38.164777 23238 tablet.cc:2369] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:12:38.203703 23412 raft_consensus.cc:482] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:12:38.203859 23412 raft_consensus.cc:504] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0395fa975b704a6ba7e1767d9d25e116" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43633 } } peers { permanent_uuid: "b675ac64627442549aa6f6b510b87bfb" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34779 } } peers { permanent_uuid: "4ca275b4eb414b44b520f529220af200" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39605 } }
I20260929 19:12:38.204285 23412 leader_election.cc:288] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0395fa975b704a6ba7e1767d9d25e116 (127.9.236.1:43633), b675ac64627442549aa6f6b510b87bfb (127.9.236.3:34779)
I20260929 19:12:38.208506 23402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:38.204657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:37173 (local address 127.9.236.1:43633)
0929 19:12:38.204697 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:38.204699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:38.204840 (+ 141us) server_negotiation.cc:407] Connection header received
0929 19:12:38.204931 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:38.204932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:38.204944 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:38.204970 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:12:38.205259 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.206018 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.207523 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.208089 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.208130 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:38.208141 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:38.208378 (+ 237us) server_negotiation.cc:299] Negotiation successful
0929 19:12:38.208425 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:12:38.208685 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:38.204641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:39607 (local address 127.9.236.3:34779)
0929 19:12:38.204691 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:38.204693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:38.204826 (+ 133us) server_negotiation.cc:407] Connection header received
0929 19:12:38.204932 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:38.204932 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:38.204942 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:38.204970 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:12:38.205259 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.206019 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.207706 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.208298 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.208340 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:38.208350 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:38.208572 (+ 222us) server_negotiation.cc:299] Negotiation successful
0929 19:12:38.208618 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 19:12:38.208961 23414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:38.204444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:34779 (local address 127.9.236.2:39607)
0929 19:12:38.204748 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:38.204759 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:38.204853 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:38.205043 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:38.205043 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:38.205044 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:12:38.205183 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:38.205186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.206115 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.206116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:38.207622 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:38.207625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.208429 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.208430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:38.208501 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:38.208503 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:12:38.208792 (+ 289us) client_negotiation.cc:240] Negotiation successful
0929 19:12:38.208841 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":55,"thread_start_us":98,"threads_started":1}
I20260929 19:12:38.209306 23325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4806f22122474ae1be634e58a4b46cec" candidate_uuid: "4ca275b4eb414b44b520f529220af200" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b675ac64627442549aa6f6b510b87bfb" is_pre_election: true
I20260929 19:12:38.209475 23325 raft_consensus.cc:2454] T 4806f22122474ae1be634e58a4b46cec P b675ac64627442549aa6f6b510b87bfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ca275b4eb414b44b520f529220af200 in term 0.
I20260929 19:12:38.209695 23126 leader_election.cc:302] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ca275b4eb414b44b520f529220af200, b675ac64627442549aa6f6b510b87bfb; no voters:
I20260929 19:12:38.209851 23412 raft_consensus.cc:2781] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:12:38.209697 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:38.204444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:43633 (local address 127.9.236.2:37173)
0929 19:12:38.204792 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:38.204799 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:38.204859 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:38.205042 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:38.205043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:38.205044 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:12:38.205183 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:38.205186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.206115 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.206116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:38.207429 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:38.207433 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.208199 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.208200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:38.208294 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:38.208297 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:12:38.208939 (+ 642us) client_negotiation.cc:240] Negotiation successful
0929 19:12:38.208981 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":71,"thread_start_us":93,"threads_started":1}
I20260929 19:12:38.209897 23412 raft_consensus.cc:482] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:12:38.209920 23412 raft_consensus.cc:3037] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:12:38.210228 23038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4806f22122474ae1be634e58a4b46cec" candidate_uuid: "4ca275b4eb414b44b520f529220af200" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0395fa975b704a6ba7e1767d9d25e116" is_pre_election: true
I20260929 19:12:38.210456 23038 raft_consensus.cc:2454] T 4806f22122474ae1be634e58a4b46cec P 0395fa975b704a6ba7e1767d9d25e116 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ca275b4eb414b44b520f529220af200 in term 0.
I20260929 19:12:38.210642 23412 raft_consensus.cc:504] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0395fa975b704a6ba7e1767d9d25e116" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43633 } } peers { permanent_uuid: "b675ac64627442549aa6f6b510b87bfb" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34779 } } peers { permanent_uuid: "4ca275b4eb414b44b520f529220af200" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39605 } }
I20260929 19:12:38.210788 23412 leader_election.cc:288] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 [CANDIDATE]: Term 1 election: Requested vote from peers 0395fa975b704a6ba7e1767d9d25e116 (127.9.236.1:43633), b675ac64627442549aa6f6b510b87bfb (127.9.236.3:34779)
I20260929 19:12:38.210919 23038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4806f22122474ae1be634e58a4b46cec" candidate_uuid: "4ca275b4eb414b44b520f529220af200" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0395fa975b704a6ba7e1767d9d25e116"
I20260929 19:12:38.211017 23038 raft_consensus.cc:3037] T 4806f22122474ae1be634e58a4b46cec P 0395fa975b704a6ba7e1767d9d25e116 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:12:38.211058 23325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4806f22122474ae1be634e58a4b46cec" candidate_uuid: "4ca275b4eb414b44b520f529220af200" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b675ac64627442549aa6f6b510b87bfb"
I20260929 19:12:38.211120 23325 raft_consensus.cc:3037] T 4806f22122474ae1be634e58a4b46cec P b675ac64627442549aa6f6b510b87bfb [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:12:38.211838 23038 raft_consensus.cc:2454] T 4806f22122474ae1be634e58a4b46cec P 0395fa975b704a6ba7e1767d9d25e116 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca275b4eb414b44b520f529220af200 in term 1.
I20260929 19:12:38.211932 23325 raft_consensus.cc:2454] T 4806f22122474ae1be634e58a4b46cec P b675ac64627442549aa6f6b510b87bfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca275b4eb414b44b520f529220af200 in term 1.
I20260929 19:12:38.212018 23125 leader_election.cc:302] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0395fa975b704a6ba7e1767d9d25e116, 4ca275b4eb414b44b520f529220af200; no voters:
I20260929 19:12:38.212271 23412 raft_consensus.cc:2781] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:12:38.212436 23412 raft_consensus.cc:686] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 [term 1 LEADER]: Becoming Leader. State: Replica: 4ca275b4eb414b44b520f529220af200, State: Running, Role: LEADER
I20260929 19:12:38.212532 23412 consensus_queue.cc:237] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0395fa975b704a6ba7e1767d9d25e116" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43633 } } peers { permanent_uuid: "b675ac64627442549aa6f6b510b87bfb" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34779 } } peers { permanent_uuid: "4ca275b4eb414b44b520f529220af200" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39605 } }
I20260929 19:12:38.213335 22905 catalog_manager.cc:5495] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ca275b4eb414b44b520f529220af200 (127.9.236.2). New cstate: current_term: 1 leader_uuid: "4ca275b4eb414b44b520f529220af200" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0395fa975b704a6ba7e1767d9d25e116" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b675ac64627442549aa6f6b510b87bfb" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca275b4eb414b44b520f529220af200" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 39605 } health_report { overall_health: HEALTHY } } }
I20260929 19:12:38.230604 23403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:38.227285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35940 (local address 127.9.236.2:39605)
0929 19:12:38.227331 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:38.227333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:38.227376 (+ 43us) server_negotiation.cc:407] Connection header received
0929 19:12:38.227502 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:38.227503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:38.227526 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:38.227580 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:12:38.227749 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.228397 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.229506 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:38.229797 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:38.229860 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:38.230046 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:12:38.230180 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:12:38.230256 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:38.230502 (+ 246us) server_negotiation.cc:299] Negotiation successful
0929 19:12:38.230524 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:12:38.232548 23038 raft_consensus.cc:1261] T 4806f22122474ae1be634e58a4b46cec P 0395fa975b704a6ba7e1767d9d25e116 [term 1 FOLLOWER]: Refusing update from remote peer 4ca275b4eb414b44b520f529220af200: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:12:38.232548 23325 raft_consensus.cc:1261] T 4806f22122474ae1be634e58a4b46cec P b675ac64627442549aa6f6b510b87bfb [term 1 FOLLOWER]: Refusing update from remote peer 4ca275b4eb414b44b520f529220af200: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:12:38.232878 23415 consensus_queue.cc:1035] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b675ac64627442549aa6f6b510b87bfb" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:12:38.233053 23412 consensus_queue.cc:1035] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0395fa975b704a6ba7e1767d9d25e116" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 43633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:12:38.235644 23418 mvcc.cc:205] Tried to move back new op lower bound from 7334744712117538816 to 7334744712039084032. Current Snapshot: MvccSnapshot[applied={T|T < 7334744712117538816}]
I20260929 19:12:38.236431 23419 mvcc.cc:205] Tried to move back new op lower bound from 7334744712117538816 to 7334744712039084032. Current Snapshot: MvccSnapshot[applied={T|T < 7334744712117538816}]
I20260929 19:12:38.238173 23420 mvcc.cc:205] Tried to move back new op lower bound from 7334744712117538816 to 7334744712039084032. Current Snapshot: MvccSnapshot[applied={T|T < 7334744712117538816}]
I20260929 19:12:38.239266 22903 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:12:38.239358 22903 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:12:38.240064 22903 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0dc377c4db39407eaaebd3cb3e841001: Sending DeleteTablet for 3 replicas of tablet 4806f22122474ae1be634e58a4b46cec
I20260929 19:12:38.240350 23155 tablet_service.cc:1514] Processing DeleteTablet for tablet 4806f22122474ae1be634e58a4b46cec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:12:38 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:35926
I20260929 19:12:38.240468 23287 tablet_service.cc:1514] Processing DeleteTablet for tablet 4806f22122474ae1be634e58a4b46cec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:12:38 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:49204
I20260929 19:12:38.240592 23026 tablet_service.cc:1514] Processing DeleteTablet for tablet 4806f22122474ae1be634e58a4b46cec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:12:38 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:35952
I20260929 19:12:38.240994 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 22943
I20260929 19:12:38.245966 23427 tablet_replica.cc:330] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200: stopping tablet replica
I20260929 19:12:38.246079 23427 raft_consensus.cc:2229] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:12:38.246179 23427 raft_consensus.cc:2258] T 4806f22122474ae1be634e58a4b46cec P 4ca275b4eb414b44b520f529220af200 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260929 19:12:38.249285 22890 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.236.1:43633 (error 108)
I20260929 19:12:38.249861 23428 tablet_replica.cc:330] T 4806f22122474ae1be634e58a4b46cec P b675ac64627442549aa6f6b510b87bfb: stopping tablet replica
I20260929 19:12:38.249989 23428 raft_consensus.cc:2229] T 4806f22122474ae1be634e58a4b46cec P b675ac64627442549aa6f6b510b87bfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:12:38.250109 23428 raft_consensus.cc:2258] T 4806f22122474ae1be634e58a4b46cec P b675ac64627442549aa6f6b510b87bfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:12:38.250301 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 23106
I20260929 19:12:38.251643 23428 ts_tablet_manager.cc:1905] T 4806f22122474ae1be634e58a4b46cec P b675ac64627442549aa6f6b510b87bfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:12:38.252831 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:38.252609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:43633 (local address 127.0.0.1:35956)
0929 19:12:38.252681 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:38.252773 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.1:43633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 19:12:38.253008 22890 catalog_manager.cc:4584] TS 0395fa975b704a6ba7e1767d9d25e116 (127.9.236.1:43633): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4806f22122474ae1be634e58a4b46cec: Network error: Client connection negotiation failed: client connection to 127.9.236.1:43633: connect: Connection refused (error 111)
I20260929 19:12:38.253937 23428 ts_tablet_manager.cc:1918] T 4806f22122474ae1be634e58a4b46cec P b675ac64627442549aa6f6b510b87bfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:12:38.254017 23428 log.cc:1196] T 4806f22122474ae1be634e58a4b46cec P b675ac64627442549aa6f6b510b87bfb: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/4806f22122474ae1be634e58a4b46cec
I20260929 19:12:38.254283 23428 ts_tablet_manager.cc:1939] T 4806f22122474ae1be634e58a4b46cec P b675ac64627442549aa6f6b510b87bfb: Deleting consensus metadata
I20260929 19:12:38.254699 22891 catalog_manager.cc:4841] TS b675ac64627442549aa6f6b510b87bfb (127.9.236.3:34779): tablet 4806f22122474ae1be634e58a4b46cec (table test-table [id=e46168b10f6d4d68a1a591f7c5e06249]) successfully deleted
I20260929 19:12:38.256747 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:38.256594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:39605 (local address 127.0.0.1:35946)
0929 19:12:38.256691 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:38.256711 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.2:39605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260929 19:12:38.256886 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 23240
I20260929 19:12:38.261502 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 22872
I20260929 19:12:38.268445 10160 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 19:12:38.602893 10160 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 19:12:38.603005 10160 mini_ranger.cc:66] Stopping Ranger...
I20260929 19:12:38.937928 10160 mini_ranger.cc:68] Stopped Ranger
2026-09-29 19:12:38.938 UTC [21939] LOG: received smart shutdown request
2026-09-29 19:12:38.942 UTC [21939] LOG: background worker "logical replication launcher" (PID 21948) exited with exit code 1
2026-09-29 19:12:38.942 UTC [21943] LOG: shutting down
2026-09-29 19:12:39.099 UTC [21939] LOG: database system is shut down
2026-09-29T19:12:39Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49677 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:12:39 dist-test-slave-wftq krb5kdc[23439](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:12:39 dist-test-slave-wftq krb5kdc[23439](info): set up 2 sockets
Sep 29 19:12:39 dist-test-slave-wftq krb5kdc[23439](info): commencing operation
krb5kdc: starting...
W20260929 19:12:40.141038 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:12:40 dist-test-slave-wftq krb5kdc[23439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:12:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:12:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:12:40.267514 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:39759
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:38871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:39759,127.9.236.61:36929,127.9.236.60:45669
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:12:40.350570 23452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:40.350796 23452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:40.350828 23452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:40.351955 23452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:12:40.351998 23452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:40.352016 23452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:40.352031 23452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:12:40.352046 23452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:12:40.353523 23452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:38871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:39759,127.9.236.61:36929,127.9.236.60:45669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:39759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:40.353781 23452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:40.353989 23452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:40.359186 23460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:40.359246 23457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:40.359246 23458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:40.359580 23452 server_base.cc:1034] running on GCE node
I20260929 19:12:40.359884 23452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:40.360241 23452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:40.361415 23452 hybrid_clock.cc:648] HybridClock initialized: now 1790709160361394 us; error 36 us; skew 500 ppm
Sep 29 19:12:40 dist-test-slave-wftq krb5kdc[23439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709160, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:40.364058 23452 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:12:40.364559 23452 webserver.cc:460] Webserver started at http://127.9.236.62:41143/ using document root <none> and password file <none>
I20260929 19:12:40.364774 23452 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:40.364825 23452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:12:40.364951 23452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:12:40.366201 23452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "52698ec68a4241f4933f13b4ebb7fe03"
format_stamp: "Formatted at 2026-09-29 19:12:40 on dist-test-slave-wftq"
server_key: "a9114b75effcac525233ae3d35c3cd33"
server_key_iv: "a9ce896ec7f369dc784fff2e8924a942"
server_key_version: "encryptionkey@0"
I20260929 19:12:40.366571 23452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "52698ec68a4241f4933f13b4ebb7fe03"
format_stamp: "Formatted at 2026-09-29 19:12:40 on dist-test-slave-wftq"
server_key: "a9114b75effcac525233ae3d35c3cd33"
server_key_iv: "a9ce896ec7f369dc784fff2e8924a942"
server_key_version: "encryptionkey@0"
I20260929 19:12:40.367960 23452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:12:40.368829 23467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:40.369041 23452 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:40.369153 23452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "52698ec68a4241f4933f13b4ebb7fe03"
format_stamp: "Formatted at 2026-09-29 19:12:40 on dist-test-slave-wftq"
server_key: "a9114b75effcac525233ae3d35c3cd33"
server_key_iv: "a9ce896ec7f369dc784fff2e8924a942"
server_key_version: "encryptionkey@0"
I20260929 19:12:40.369226 23452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:40.377569 23452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:40.379803 23452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:40.379942 23452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:40.384598 23452 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:39759
I20260929 19:12:40.384652 23519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:39759 every 8 connection(s)
I20260929 19:12:40.385007 23452 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:12:40.386018 23520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:12:40.387631 23520 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:12:40.393122 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 23452
I20260929 19:12:40.393234 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:12:40.393450 10160 external_mini_cluster.cc:1428] Setting key 833b615fc5d68678781984171fe9e719
Sep 29 19:12:40 dist-test-slave-wftq krb5kdc[23439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709160, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:12:40.397958 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.388363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39759 (local address 127.0.0.1:58254)
0929 19:12:40.388549 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.388587 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:40.389042 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:40.389419 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:40.389422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:40.389680 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:40.389849 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:40.389853 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.390924 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.390924 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:40.391651 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:40.391653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.392214 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.392214 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:40.392282 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.392851 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:40.392859 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:40.394295 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:40.396333 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:40.396340 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:40.396341 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:40.396551 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:40.396827 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:40.396828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:40.396828 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:40.396904 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:40.397390 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:40.397394 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:40.397454 (+ 60us) client_negotiation.cc:769] Sending connection context
0929 19:12:40.397537 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 19:12:40.397718 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":77,"threads_started":1}
I20260929 19:12:40.397960 23523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.388560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58254 (local address 127.9.236.62:39759)
0929 19:12:40.388700 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:40.388704 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:40.388735 (+ 31us) server_negotiation.cc:407] Connection header received
0929 19:12:40.389243 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:40.389245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:40.389275 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:40.389370 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:40.389952 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.390841 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.391754 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.392131 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.392235 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.394392 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:40.394402 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:40.394402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:40.394411 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:40.396225 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.396631 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.396634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.396635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.396754 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.396967 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.396968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.396968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.397299 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:40.397402 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:40.397598 (+ 196us) server_negotiation.cc:299] Negotiation successful
0929 19:12:40.397790 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":108,"thread_start_us":52,"threads_started":1}
I20260929 19:12:40.398802 23520 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:12:40.399123 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.399022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:36929 (local address 127.0.0.1:50798)
0929 19:12:40.399062 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.399104 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.61:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260929 19:12:40.399251 23470 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.236.61:36929: connect: Connection refused (error 111)
I20260929 19:12:40.399609 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.399532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:36929 (local address 127.0.0.1:50808)
0929 19:12:40.399566 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.399588 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.61:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260929 19:12:40.399739 23520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.61:36929: Network error: Client connection negotiation failed: client connection to 127.9.236.61:36929: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.9.236.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.61.keytab.
Entry for principal kudu/127.9.236.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.61.keytab.
I20260929 19:12:40.420325 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.61:36929
--webserver_interface=127.9.236.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.61.keytab
--principal=kudu/127.9.236.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:38871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:39759,127.9.236.61:36929,127.9.236.60:45669
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:12:40.448953 23520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } attempt: 1
I20260929 19:12:40.449570 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.449415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:36929 (local address 127.0.0.1:50812)
0929 19:12:40.449491 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.449544 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.61:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260929 19:12:40.449944 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.449851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:36929 (local address 127.0.0.1:50816)
0929 19:12:40.449900 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.449927 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.61:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 19:12:40.450053 23520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.61:36929: Network error: Client connection negotiation failed: client connection to 127.9.236.61:36929: connect: Connection refused (error 111)
W20260929 19:12:40.507611 23527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:40.507848 23527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:40.507869 23527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:40.509047 23527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:12:40.509087 23527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:40.509125 23527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:40.509155 23527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:12:40.509171 23527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:12:40.510661 23527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:38871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:39759,127.9.236.61:36929,127.9.236.60:45669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.61.keytab
--principal=kudu/127.9.236.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.61:36929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.236.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:40.510916 23527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:40.511154 23527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:40.516139 23535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:40.516147 23532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:40.516430 23527 server_base.cc:1034] running on GCE node
W20260929 19:12:40.516189 23533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:40.516768 23527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:40.517079 23527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:40.518177 23520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } attempt: 2
I20260929 19:12:40.518252 23527 hybrid_clock.cc:648] HybridClock initialized: now 1790709160518233 us; error 35 us; skew 500 ppm
I20260929 19:12:40.518678 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.518507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:36929 (local address 127.0.0.1:50820)
0929 19:12:40.518588 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.518640 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.61:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260929 19:12:40.519088 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.518949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:36929 (local address 127.0.0.1:50826)
0929 19:12:40.519007 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.519047 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.61:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260929 19:12:40.519204 23520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.61:36929: Network error: Client connection negotiation failed: client connection to 127.9.236.61:36929: connect: Connection refused (error 111)
Sep 29 19:12:40 dist-test-slave-wftq krb5kdc[23439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709160, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:40.524633 23527 init.cc:377] Logged in from keytab as kudu/127.9.236.61@KRBTEST.COM (short username kudu)
I20260929 19:12:40.525198 23527 webserver.cc:460] Webserver started at http://127.9.236.61:43619/ using document root <none> and password file <none>
I20260929 19:12:40.525462 23527 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:40.525525 23527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:12:40.525671 23527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:12:40.526928 23527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-1/data/instance:
uuid: "949d544dddc640478e860fa73f0afbbc"
format_stamp: "Formatted at 2026-09-29 19:12:40 on dist-test-slave-wftq"
server_key: "d64a7fe35574243ef8d4485825d490e9"
server_key_iv: "af43a4cfb48a3951e09dbcf910f842d3"
server_key_version: "encryptionkey@0"
I20260929 19:12:40.527321 23527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-1/wal/instance:
uuid: "949d544dddc640478e860fa73f0afbbc"
format_stamp: "Formatted at 2026-09-29 19:12:40 on dist-test-slave-wftq"
server_key: "d64a7fe35574243ef8d4485825d490e9"
server_key_iv: "af43a4cfb48a3951e09dbcf910f842d3"
server_key_version: "encryptionkey@0"
I20260929 19:12:40.528762 23527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:12:40.529671 23542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:40.529861 23527 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 19:12:40.529955 23527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-1/wal
uuid: "949d544dddc640478e860fa73f0afbbc"
format_stamp: "Formatted at 2026-09-29 19:12:40 on dist-test-slave-wftq"
server_key: "d64a7fe35574243ef8d4485825d490e9"
server_key_iv: "af43a4cfb48a3951e09dbcf910f842d3"
server_key_version: "encryptionkey@0"
I20260929 19:12:40.530030 23527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:40.554071 23527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:40.556196 23527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:40.556330 23527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:40.560876 23527 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.61:36929
I20260929 19:12:40.560961 23594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.61:36929 every 8 connection(s)
I20260929 19:12:40.561305 23527 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-1/data/info.pb
I20260929 19:12:40.562186 23595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:12:40.563763 23595 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:12:40.566141 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 23527
I20260929 19:12:40.566262 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-1/wal/instance
I20260929 19:12:40.566520 10160 external_mini_cluster.cc:1428] Setting key fc6055c97f5e0e14d2fe62720ffebac3
Sep 29 19:12:40 dist-test-slave-wftq krb5kdc[23439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709160, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.61@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:12:40.573156 23523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.564503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58262 (local address 127.9.236.62:39759)
0929 19:12:40.564553 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:40.564555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:40.564783 (+ 228us) server_negotiation.cc:407] Connection header received
0929 19:12:40.565303 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:40.565305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:40.565325 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:40.565383 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:40.565994 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.566756 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.567753 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.568066 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.568133 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.570566 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:40.570568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:40.570568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:40.570575 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:40.571858 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.572243 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.572243 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.572244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.572290 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.572491 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.572492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.572492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.572608 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:40.572661 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:40.572930 (+ 269us) server_negotiation.cc:299] Negotiation successful
0929 19:12:40.573019 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 19:12:40.573285 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.564479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39759 (local address 127.0.0.1:58262)
0929 19:12:40.564649 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.564685 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:40.565171 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:40.565449 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:40.565452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:40.565698 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:40.565892 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:40.565895 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.566859 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.566860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:40.567663 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:40.567665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.568152 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.568153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:40.568295 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.568884 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:40.568902 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:40.570465 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:40.571951 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:40.571958 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:40.571961 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:40.572179 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:40.572341 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:40.572342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:40.572342 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:40.572439 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:40.572660 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:40.572683 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:40.572812 (+ 129us) client_negotiation.cc:769] Sending connection context
0929 19:12:40.572904 (+ 92us) client_negotiation.cc:240] Negotiation successful
0929 19:12:40.573045 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
I20260929 19:12:40.573748 23595 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.236.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.60@KRBTEST.COM" created.
Sep 29 19:12:40 dist-test-slave-wftq krb5kdc[23439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709160, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.61@KRBTEST.COM for kudu/127.9.236.61@KRBTEST.COM
I20260929 19:12:40.580922 23599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.574127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50840 (local address 127.9.236.61:36929)
0929 19:12:40.574316 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:40.574320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:40.574334 (+ 14us) server_negotiation.cc:407] Connection header received
0929 19:12:40.574349 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:40.574350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:40.574361 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:40.574425 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:40.574682 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.575345 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.575966 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.576250 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.576305 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.577986 (+ 1681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:40.577994 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:40.577995 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:40.578001 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:40.579564 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.580088 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.580092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.580093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.580156 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.580392 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.580393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.580393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.580527 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:40.580591 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:40.580754 (+ 163us) server_negotiation.cc:299] Negotiation successful
0929 19:12:40.580838 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":123,"threads_started":1}
I20260929 19:12:40.581012 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.573993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:36929 (local address 127.0.0.1:50840)
0929 19:12:40.574032 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.574042 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:40.574076 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:40.574436 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:40.574436 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:40.574582 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:40.574623 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:40.574625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.575426 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.575427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:40.575902 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:40.575903 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.576309 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.576310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:40.576366 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.576800 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:40.576803 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:40.577916 (+ 1113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:40.579692 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:40.579694 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:40.579695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:40.579955 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:40.580236 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:40.580237 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:40.580237 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:40.580311 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:40.580589 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:40.580590 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:40.580654 (+ 64us) client_negotiation.cc:769] Sending connection context
0929 19:12:40.580720 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 19:12:40.580903 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":65}
I20260929 19:12:40.581589 23595 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:12:40.581910 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.581792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:45669 (local address 127.0.0.1:53150)
0929 19:12:40.581838 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.581888 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 19:12:40.582011 23545 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.236.60:45669: connect: Connection refused (error 111)
I20260929 19:12:40.582384 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.582298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:45669 (local address 127.0.0.1:53156)
0929 19:12:40.582332 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.582354 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260929 19:12:40.582471 23595 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.60:45669: Network error: Client connection negotiation failed: client connection to 127.9.236.60:45669: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.60.keytab.
Entry for principal kudu/127.9.236.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.60.keytab.
I20260929 19:12:40.592839 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.60:45669
--webserver_interface=127.9.236.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.60.keytab
--principal=kudu/127.9.236.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:38871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:39759,127.9.236.61:36929,127.9.236.60:45669
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:12:40.610368 23520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } attempt: 3
Sep 29 19:12:40 dist-test-slave-wftq krb5kdc[23439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709160, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for kudu/127.9.236.61@KRBTEST.COM
I20260929 19:12:40.617432 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.610776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:36929 (local address 127.0.0.1:50854)
0929 19:12:40.610840 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.610852 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:40.610900 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:40.611045 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:40.611046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:40.611218 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:40.611294 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:40.611296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.611998 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.611999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:40.612492 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:40.612493 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.612831 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.612832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:40.612901 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.613414 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:40.613419 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:40.614864 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:40.616419 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:40.616420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:40.616420 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:40.616602 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:40.616776 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:40.616777 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:40.616778 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:40.616834 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:40.617063 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:40.617064 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:40.617137 (+ 73us) client_negotiation.cc:769] Sending connection context
0929 19:12:40.617201 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 19:12:40.617317 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260929 19:12:40.617432 23599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.610856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50854 (local address 127.9.236.61:36929)
0929 19:12:40.610933 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:40.610935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:40.610945 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:12:40.610975 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:40.610977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:40.610990 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:40.611035 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:40.611365 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.611924 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.612561 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.612775 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.612830 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.614944 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:40.614945 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:40.614946 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:40.614953 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:40.616329 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.616668 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.616669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.616670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.616724 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.616883 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.616883 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.616884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.617002 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:40.617067 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:40.617231 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 19:12:40.617317 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 19:12:40.617883 23520 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:12:40.618302 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.618198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:45669 (local address 127.0.0.1:53168)
0929 19:12:40.618254 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.618285 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260929 19:12:40.618724 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.618617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:45669 (local address 127.0.0.1:53176)
0929 19:12:40.618678 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.618702 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 19:12:40.618809 23520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.60:45669: Network error: Client connection negotiation failed: client connection to 127.9.236.60:45669: connect: Connection refused (error 111)
I20260929 19:12:40.631681 23595 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } attempt: 1
I20260929 19:12:40.632190 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.632017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:45669 (local address 127.0.0.1:53182)
0929 19:12:40.632105 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.632158 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260929 19:12:40.632531 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.632440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:45669 (local address 127.0.0.1:53192)
0929 19:12:40.632482 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.632513 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260929 19:12:40.632618 23595 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.60:45669: Network error: Client connection negotiation failed: client connection to 127.9.236.60:45669: connect: Connection refused (error 111)
I20260929 19:12:40.649947 23520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } attempt: 1
I20260929 19:12:40.650408 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.650288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:45669 (local address 127.0.0.1:53202)
0929 19:12:40.650345 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.650379 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 19:12:40.650867 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.650784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:45669 (local address 127.0.0.1:53210)
0929 19:12:40.650827 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.650851 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260929 19:12:40.650983 23520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.60:45669: Network error: Client connection negotiation failed: client connection to 127.9.236.60:45669: connect: Connection refused (error 111)
W20260929 19:12:40.678700 23602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:40.678925 23602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:40.678946 23602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:40.680154 23602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:12:40.680208 23602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:40.680233 23602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:40.680251 23602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:12:40.680272 23602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:12:40.682389 23602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:38871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:39759,127.9.236.61:36929,127.9.236.60:45669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.60.keytab
--principal=kudu/127.9.236.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.60:45669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.236.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:40.682651 23602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:40.682902 23602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:40.688652 23607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:40.688809 23608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:40.689132 23610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:40.689558 23602 server_base.cc:1034] running on GCE node
I20260929 19:12:40.689834 23602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:40.690137 23602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:40.691383 23602 hybrid_clock.cc:648] HybridClock initialized: now 1790709160691351 us; error 42 us; skew 500 ppm
Sep 29 19:12:40 dist-test-slave-wftq krb5kdc[23439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709160, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:40.698407 23602 init.cc:377] Logged in from keytab as kudu/127.9.236.60@KRBTEST.COM (short username kudu)
I20260929 19:12:40.699014 23602 webserver.cc:460] Webserver started at http://127.9.236.60:35089/ using document root <none> and password file <none>
I20260929 19:12:40.699272 23602 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:40.699349 23602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:12:40.699474 23602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:12:40.700755 23595 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } attempt: 2
I20260929 19:12:40.700946 23602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-2/data/instance:
uuid: "ef6365b7155549aabf35feef089324fc"
format_stamp: "Formatted at 2026-09-29 19:12:40 on dist-test-slave-wftq"
server_key: "2df82480f99b5126b3f3aec11ccc749f"
server_key_iv: "28f0844994cfe62b52cbe4732151ae2d"
server_key_version: "encryptionkey@0"
I20260929 19:12:40.701294 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.701164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:45669 (local address 127.0.0.1:53212)
0929 19:12:40.701222 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.701257 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260929 19:12:40.701412 23602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-2/wal/instance:
uuid: "ef6365b7155549aabf35feef089324fc"
format_stamp: "Formatted at 2026-09-29 19:12:40 on dist-test-slave-wftq"
server_key: "2df82480f99b5126b3f3aec11ccc749f"
server_key_iv: "28f0844994cfe62b52cbe4732151ae2d"
server_key_version: "encryptionkey@0"
I20260929 19:12:40.701613 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.701518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:45669 (local address 127.0.0.1:53216)
0929 19:12:40.701569 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.701593 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 19:12:40.701727 23595 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.60:45669: Network error: Client connection negotiation failed: client connection to 127.9.236.60:45669: connect: Connection refused (error 111)
I20260929 19:12:40.703215 23602 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260929 19:12:40.704164 23617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:40.704509 23602 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:40.704613 23602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-2/wal
uuid: "ef6365b7155549aabf35feef089324fc"
format_stamp: "Formatted at 2026-09-29 19:12:40 on dist-test-slave-wftq"
server_key: "2df82480f99b5126b3f3aec11ccc749f"
server_key_iv: "28f0844994cfe62b52cbe4732151ae2d"
server_key_version: "encryptionkey@0"
I20260929 19:12:40.704696 23602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:40.718345 23602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:40.720736 23602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:40.720896 23602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:40.726115 23520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } attempt: 2
I20260929 19:12:40.726622 23602 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.60:45669
I20260929 19:12:40.726658 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.726513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:45669 (local address 127.0.0.1:53218)
0929 19:12:40.726580 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.726618 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 19:12:40.726670 23669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.60:45669 every 8 connection(s)
I20260929 19:12:40.727099 23602 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-2/data/info.pb
I20260929 19:12:40.728087 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 23602
I20260929 19:12:40.728170 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790709008521303-10160-0/minicluster-data/master-2/wal/instance
I20260929 19:12:40.728396 10160 external_mini_cluster.cc:1428] Setting key 07d20eaad3b17b0c99d984eb36e65eb5
I20260929 19:12:40.728645 23670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:12:40.730729 23670 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 19:12:40 dist-test-slave-wftq krb5kdc[23439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709160, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for kudu/127.9.236.60@KRBTEST.COM
Sep 29 19:12:40 dist-test-slave-wftq krb5kdc[23439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709160, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.60@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
Sep 29 19:12:40 dist-test-slave-wftq krb5kdc[23439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.60@KRBTEST.COM
I20260929 19:12:40.739642 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.727087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:45669 (local address 127.0.0.1:53228)
0929 19:12:40.727121 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.727131 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:40.727189 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:40.729049 (+ 1860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:40.729050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:40.729498 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:40.729560 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:40.729565 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.731257 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.731258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:40.731889 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:40.731891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.732743 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.732744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:40.732809 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.733453 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:40.733459 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:40.734873 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:40.737219 (+ 2346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:40.737221 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:40.737221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:40.737466 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:40.738672 (+ 1206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:40.738672 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:40.738673 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:40.738745 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:40.739195 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:40.739196 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:40.739252 (+ 56us) client_negotiation.cc:769] Sending connection context
0929 19:12:40.739410 (+ 158us) client_negotiation.cc:240] Negotiation successful
0929 19:12:40.739524 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260929 19:12:40.739867 23523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.731594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58278 (local address 127.9.236.62:39759)
0929 19:12:40.731689 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:40.731692 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:40.731700 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:12:40.731719 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:40.731720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:40.731730 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:40.731792 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:40.732189 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.732674 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.733776 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.733992 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.734048 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.736983 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:40.736985 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:40.736985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:40.736993 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:40.738576 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.739014 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.739015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.739015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.739068 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.739294 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.739295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.739295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.739425 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:40.739482 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:40.739660 (+ 178us) server_negotiation.cc:299] Negotiation successful
0929 19:12:40.739764 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 19:12:40.740262 23675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.731326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:39759 (local address 127.0.0.1:58278)
0929 19:12:40.731495 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.731530 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:40.731583 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:40.731807 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:40.731809 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:40.732043 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:40.732115 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:40.732118 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.732767 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.732768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:40.733712 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:40.733714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.734060 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.734061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:40.734137 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.734978 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:40.734986 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:40.736878 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:40.738671 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:40.738676 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:40.738678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:40.738938 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:40.739141 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:40.739141 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:40.739142 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:40.739203 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:40.739482 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:40.739489 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:40.739552 (+ 63us) client_negotiation.cc:769] Sending connection context
0929 19:12:40.739631 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 19:12:40.739962 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":115,"threads_started":1}
I20260929 19:12:40.740270 23672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.727332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53228 (local address 127.9.236.60:45669)
0929 19:12:40.728006 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:40.728017 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:40.728062 (+ 45us) server_negotiation.cc:407] Connection header received
0929 19:12:40.728684 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:40.728693 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:40.728759 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:40.729073 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:40.729679 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.731166 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.731957 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.732639 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.733003 (+ 364us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.734965 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:40.734973 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:40.734973 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:40.734988 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:40.737070 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.737818 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.737822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.737823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.737906 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.738834 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.738835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.738836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.739132 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:40.739247 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:40.739796 (+ 549us) server_negotiation.cc:299] Negotiation successful
0929 19:12:40.740021 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":106,"server-negotiator.queue_time_us":602,"thread_start_us":533,"threads_started":1}
I20260929 19:12:40.740936 23674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.728967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53240 (local address 127.9.236.60:45669)
0929 19:12:40.731180 (+ 2213us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:40.731183 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:40.731192 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:12:40.731211 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:40.731212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:40.731220 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:40.731291 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:40.732736 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.733551 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.734546 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.734846 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.734910 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.737562 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:40.737563 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:40.737564 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:40.737572 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:40.739702 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.740125 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.740126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.740126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.740191 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.740410 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.740411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.740412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.740524 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:40.740653 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:40.740729 (+ 76us) server_negotiation.cc:299] Negotiation successful
0929 19:12:40.740806 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2194,"thread_start_us":71,"threads_started":1}
I20260929 19:12:40.740926 23670 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:12:40.742545 23520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03: Bootstrap starting.
I20260929 19:12:40.743228 23520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03: Neither blocks nor log segments found. Creating new log.
I20260929 19:12:40.743537 23520 log.cc:824] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03: Log is configured to *not* fsync() on all Append() calls
I20260929 19:12:40.744563 23520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03: No bootstrap required, opened a new log
Sep 29 19:12:40 dist-test-slave-wftq krb5kdc[23439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709160, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.60@KRBTEST.COM for kudu/127.9.236.61@KRBTEST.COM
I20260929 19:12:40.746810 23520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52698ec68a4241f4933f13b4ebb7fe03" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } } peers { permanent_uuid: "949d544dddc640478e860fa73f0afbbc" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } } peers { permanent_uuid: "ef6365b7155549aabf35feef089324fc" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } }
I20260929 19:12:40.746961 23520 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:12:40.746995 23520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52698ec68a4241f4933f13b4ebb7fe03, State: Initialized, Role: FOLLOWER
I20260929 19:12:40.747215 23520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52698ec68a4241f4933f13b4ebb7fe03" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } } peers { permanent_uuid: "949d544dddc640478e860fa73f0afbbc" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } } peers { permanent_uuid: "ef6365b7155549aabf35feef089324fc" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } }
I20260929 19:12:40.747546 23678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52698ec68a4241f4933f13b4ebb7fe03" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } } peers { permanent_uuid: "949d544dddc640478e860fa73f0afbbc" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } } peers { permanent_uuid: "ef6365b7155549aabf35feef089324fc" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } } }
I20260929 19:12:40.747632 23520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:12:40.748595 23599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.741397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50860 (local address 127.9.236.61:36929)
0929 19:12:40.741456 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:40.741458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:40.741526 (+ 68us) server_negotiation.cc:407] Connection header received
0929 19:12:40.741602 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:40.741603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:40.741613 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:40.741661 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:40.742002 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.742489 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.743441 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.743650 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.743719 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.745714 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:40.745717 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:40.745717 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:40.745726 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:40.747291 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.747662 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.747663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.747663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.747725 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.747976 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.747978 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.747978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.748144 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:40.748228 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:40.748409 (+ 181us) server_negotiation.cc:299] Negotiation successful
0929 19:12:40.748506 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 19:12:40.748631 23676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.741191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:36929 (local address 127.0.0.1:50860)
0929 19:12:40.741397 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.741409 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:40.741532 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:40.741682 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:40.741682 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:40.741840 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:40.741928 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:40.741930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.742571 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.742572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:40.743312 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:40.743315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.743719 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.743721 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:40.743801 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.744199 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:40.744204 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:40.745630 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:40.747377 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:40.747378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:40.747379 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:40.747599 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:40.747802 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:40.747803 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:40.747804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:40.747871 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:40.748238 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:40.748240 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:40.748291 (+ 51us) client_negotiation.cc:769] Sending connection context
0929 19:12:40.748380 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 19:12:40.748512 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":86,"threads_started":1}
Sep 29 19:12:40 dist-test-slave-wftq krb5kdc[23439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:12:40.747623 23678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:12:40.749409 23670 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } has no permanent_uuid. Determining permanent_uuid...
W20260929 19:12:40.749801 23689 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:12:40.749859 23689 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:12:40.750746 23452 master_runner.cc:186] Error getting master registration for 127.9.236.61:36929: OK, instance_id { permanent_uuid: "949d544dddc640478e860fa73f0afbbc" instance_seqno: 1790709160559647 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 19:12:40.753302 23452 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 29 19:12:40 dist-test-slave-wftq krb5kdc[23439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709160, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.60@KRBTEST.COM for kudu/127.9.236.60@KRBTEST.COM
I20260929 19:12:40.756654 23523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.744489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58286 (local address 127.9.236.62:39759)
0929 19:12:40.744531 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:40.744532 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:40.744568 (+ 36us) server_negotiation.cc:407] Connection header received
0929 19:12:40.744625 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:40.744626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:40.744635 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:40.744678 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:40.745280 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.745952 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.746797 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.747009 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.747062 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.749624 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:40.749625 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:40.749626 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:40.749632 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:40.755035 (+ 5403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.755653 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.755654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.755655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.755722 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.755966 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.755967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.755967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.756105 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:40.756235 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:40.756473 (+ 238us) server_negotiation.cc:299] Negotiation successful
0929 19:12:40.756555 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:12:40.757647 23676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.749622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:45669 (local address 127.0.0.1:53256)
0929 19:12:40.749698 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.749707 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:40.749738 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:40.750102 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:40.750103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:40.750255 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:40.750316 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:40.750318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.751033 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.751034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:40.751797 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:40.751799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.752258 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.752259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:40.752336 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.753870 (+ 1534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:40.753875 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:40.755116 (+ 1241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:40.756616 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:40.756617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:40.756617 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:40.756799 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:40.756975 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:40.756975 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:40.756976 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:40.757024 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:40.757353 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:40.757353 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:40.757391 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 19:12:40.757447 (+ 56us) client_negotiation.cc:240] Negotiation successful
0929 19:12:40.757550 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260929 19:12:40.757962 23674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.749925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53256 (local address 127.9.236.60:45669)
0929 19:12:40.749977 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:40.749979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:40.749987 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:12:40.750005 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:40.750006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:40.750016 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:40.750072 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:40.750377 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.750945 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.751888 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.752175 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.752245 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.755210 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:40.755211 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:40.755212 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:40.755218 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:40.756535 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.756861 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.756862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.756862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.756909 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.757076 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.757076 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.757077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.757274 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:40.757329 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:40.757811 (+ 482us) server_negotiation.cc:299] Negotiation successful
0929 19:12:40.757887 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260929 19:12:40.759207 23670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc: Bootstrap starting.
I20260929 19:12:40.760004 23670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc: Neither blocks nor log segments found. Creating new log.
I20260929 19:12:40.760274 23670 log.cc:824] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc: Log is configured to *not* fsync() on all Append() calls
I20260929 19:12:40.761222 23670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc: No bootstrap required, opened a new log
Sep 29 19:12:40 dist-test-slave-wftq krb5kdc[23439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.61@KRBTEST.COM
I20260929 19:12:40.763824 23670 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52698ec68a4241f4933f13b4ebb7fe03" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } } peers { permanent_uuid: "949d544dddc640478e860fa73f0afbbc" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } } peers { permanent_uuid: "ef6365b7155549aabf35feef089324fc" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } }
I20260929 19:12:40.763976 23670 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:12:40.764009 23670 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef6365b7155549aabf35feef089324fc, State: Initialized, Role: FOLLOWER
I20260929 19:12:40.764253 23670 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52698ec68a4241f4933f13b4ebb7fe03" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } } peers { permanent_uuid: "949d544dddc640478e860fa73f0afbbc" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } } peers { permanent_uuid: "ef6365b7155549aabf35feef089324fc" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } }
I20260929 19:12:40.764659 23670 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:12:40.764676 23599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.757371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50864 (local address 127.9.236.61:36929)
0929 19:12:40.757417 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:40.757419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:40.757429 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:12:40.757480 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:40.757481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:40.757490 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:40.757555 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:40.758270 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.758992 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.759888 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.760167 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.760241 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.762350 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:40.762353 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:40.762353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:40.762361 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:40.763654 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.763936 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.763937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.763938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.763982 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.764142 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.764143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.764143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.764261 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:40.764420 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:40.764454 (+ 34us) server_negotiation.cc:299] Negotiation successful
0929 19:12:40.764526 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 19:12:40.765297 23692 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52698ec68a4241f4933f13b4ebb7fe03" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } } peers { permanent_uuid: "949d544dddc640478e860fa73f0afbbc" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } } peers { permanent_uuid: "ef6365b7155549aabf35feef089324fc" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } } }
I20260929 19:12:40.765484 23692 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:12:40.766207 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260929 19:12:40.766539 23703 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:12:40.766584 23703 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:12:40.768039 23602 master_runner.cc:186] Error getting master registration for 127.9.236.61:36929: OK, instance_id { permanent_uuid: "949d544dddc640478e860fa73f0afbbc" instance_seqno: 1790709160559647 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 19:12:40.770267 23602 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 19:12:40.774780 23599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.767836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50876 (local address 127.9.236.61:36929)
0929 19:12:40.767871 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:40.767873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:40.768122 (+ 249us) server_negotiation.cc:407] Connection header received
0929 19:12:40.768210 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:40.768211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:40.768222 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:40.768249 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:40.768970 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.769729 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.770798 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.770996 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.771049 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.772447 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:40.772448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:40.772449 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:40.772454 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:40.773777 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.774104 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.774105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.774105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.774161 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.774323 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.774324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.774324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.774427 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:40.774533 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:40.774622 (+ 89us) server_negotiation.cc:299] Negotiation successful
0929 19:12:40.774688 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 19:12:40.775967 23674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.768187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53266 (local address 127.9.236.60:45669)
0929 19:12:40.768220 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:40.768222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:40.768421 (+ 199us) server_negotiation.cc:407] Connection header received
0929 19:12:40.768504 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:40.768505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:40.768517 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:40.768543 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:40.769179 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.769846 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.770754 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.771071 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.771150 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.772516 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:40.772517 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:40.772518 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:40.772524 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:40.774641 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.775075 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.775076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.775077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.775130 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.775351 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.775352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.775353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.775507 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:40.775623 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:40.775796 (+ 173us) server_negotiation.cc:299] Negotiation successful
0929 19:12:40.775878 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:12:40.781548 23523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.767962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58296 (local address 127.9.236.62:39759)
0929 19:12:40.768005 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:40.768007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:40.774841 (+ 6834us) server_negotiation.cc:407] Connection header received
0929 19:12:40.775216 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:40.775217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:40.775228 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:40.775274 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:40.775680 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.776217 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.776935 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.777193 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.777263 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.778646 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:40.778647 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:40.778648 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:40.778655 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:40.780289 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.780677 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.780677 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.780678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.780732 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.780936 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.780937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.780937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.781067 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:40.781215 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:40.781360 (+ 145us) server_negotiation.cc:299] Negotiation successful
0929 19:12:40.781442 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:12:40.792883 23595 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } attempt: 3
Sep 29 19:12:40 dist-test-slave-wftq krb5kdc[23439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709160, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.61@KRBTEST.COM for kudu/127.9.236.60@KRBTEST.COM
I20260929 19:12:40.801278 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.793235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:45669 (local address 127.0.0.1:53278)
0929 19:12:40.793289 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:40.793300 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:40.793334 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:40.793489 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:40.793491 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:40.793676 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:40.793740 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:40.793742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.794643 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.794644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:40.795516 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:40.795518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.796054 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.796055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:40.796111 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.796589 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:40.796594 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:40.798261 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:40.800052 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:40.800053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:40.800054 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:40.800323 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:40.800556 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:40.800557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:40.800557 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:40.800619 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:40.800903 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:40.800904 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:40.800962 (+ 58us) client_negotiation.cc:769] Sending connection context
0929 19:12:40.801038 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 19:12:40.801174 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260929 19:12:40.801291 23674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:40.793281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53278 (local address 127.9.236.60:45669)
0929 19:12:40.793321 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:40.793324 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:40.793334 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:12:40.793398 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:40.793398 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:40.793409 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:40.793477 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:40.793814 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.794561 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.795615 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:40.795935 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:40.795994 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:40.798371 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:40.798373 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:40.798374 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:40.798381 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:40.799958 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.800422 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.800423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.800424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.800483 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:40.800686 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:40.800686 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:40.800687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:40.800829 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:40.800902 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:40.801071 (+ 169us) server_negotiation.cc:299] Negotiation successful
0929 19:12:40.801184 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:12:40.803794 23595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc: Bootstrap starting.
I20260929 19:12:40.804669 23595 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc: Neither blocks nor log segments found. Creating new log.
I20260929 19:12:40.805012 23595 log.cc:824] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc: Log is configured to *not* fsync() on all Append() calls
I20260929 19:12:40.806126 23595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc: No bootstrap required, opened a new log
I20260929 19:12:40.808830 23595 raft_consensus.cc:348] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52698ec68a4241f4933f13b4ebb7fe03" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } } peers { permanent_uuid: "949d544dddc640478e860fa73f0afbbc" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } } peers { permanent_uuid: "ef6365b7155549aabf35feef089324fc" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } }
I20260929 19:12:40.808988 23595 raft_consensus.cc:374] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:12:40.809022 23595 raft_consensus.cc:729] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 949d544dddc640478e860fa73f0afbbc, State: Initialized, Role: FOLLOWER
I20260929 19:12:40.809275 23595 consensus_queue.cc:260] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52698ec68a4241f4933f13b4ebb7fe03" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } } peers { permanent_uuid: "949d544dddc640478e860fa73f0afbbc" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } } peers { permanent_uuid: "ef6365b7155549aabf35feef089324fc" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } }
I20260929 19:12:40.809660 23595 sys_catalog.cc:564] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:12:40.809625 23713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52698ec68a4241f4933f13b4ebb7fe03" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } } peers { permanent_uuid: "949d544dddc640478e860fa73f0afbbc" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } } peers { permanent_uuid: "ef6365b7155549aabf35feef089324fc" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } } }
I20260929 19:12:40.809852 23713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc [sys.catalog]: This master's current role is: FOLLOWER
W20260929 19:12:40.811705 23724 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:12:40.811816 23724 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:12:40.813601 23527 master_runner.cc:263] No leader master found from master 949d544dddc640478e860fa73f0afbbc
I20260929 19:12:40.813657 23527 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 19:12:40.859499 23678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:12:40.859669 23678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52698ec68a4241f4933f13b4ebb7fe03" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } } peers { permanent_uuid: "949d544dddc640478e860fa73f0afbbc" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } } peers { permanent_uuid: "ef6365b7155549aabf35feef089324fc" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } }
I20260929 19:12:40.859920 23678 leader_election.cc:288] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 949d544dddc640478e860fa73f0afbbc (127.9.236.61:36929), ef6365b7155549aabf35feef089324fc (127.9.236.60:45669)
I20260929 19:12:40.860488 23569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "52698ec68a4241f4933f13b4ebb7fe03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "949d544dddc640478e860fa73f0afbbc" is_pre_election: true
I20260929 19:12:40.860662 23569 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52698ec68a4241f4933f13b4ebb7fe03 in term 0.
I20260929 19:12:40.861146 23644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "52698ec68a4241f4933f13b4ebb7fe03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef6365b7155549aabf35feef089324fc" is_pre_election: true
I20260929 19:12:40.861336 23644 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52698ec68a4241f4933f13b4ebb7fe03 in term 0.
I20260929 19:12:40.861469 23470 leader_election.cc:302] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52698ec68a4241f4933f13b4ebb7fe03, 949d544dddc640478e860fa73f0afbbc; no voters:
I20260929 19:12:40.861609 23678 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:12:40.861658 23678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:12:40.861678 23678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:12:40.862737 23678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52698ec68a4241f4933f13b4ebb7fe03" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } } peers { permanent_uuid: "949d544dddc640478e860fa73f0afbbc" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } } peers { permanent_uuid: "ef6365b7155549aabf35feef089324fc" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } }
I20260929 19:12:40.862869 23678 leader_election.cc:288] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [CANDIDATE]: Term 1 election: Requested vote from peers 949d544dddc640478e860fa73f0afbbc (127.9.236.61:36929), ef6365b7155549aabf35feef089324fc (127.9.236.60:45669)
I20260929 19:12:40.863189 23569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "52698ec68a4241f4933f13b4ebb7fe03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "949d544dddc640478e860fa73f0afbbc"
I20260929 19:12:40.863279 23569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:12:40.864253 23569 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52698ec68a4241f4933f13b4ebb7fe03 in term 1.
I20260929 19:12:40.864493 23644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "52698ec68a4241f4933f13b4ebb7fe03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef6365b7155549aabf35feef089324fc"
I20260929 19:12:40.864580 23644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:12:40.865165 23470 leader_election.cc:302] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52698ec68a4241f4933f13b4ebb7fe03, 949d544dddc640478e860fa73f0afbbc; no voters:
I20260929 19:12:40.866161 23644 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52698ec68a4241f4933f13b4ebb7fe03 in term 1.
I20260929 19:12:40.866463 23678 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:12:40.866688 23678 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [term 1 LEADER]: Becoming Leader. State: Replica: 52698ec68a4241f4933f13b4ebb7fe03, State: Running, Role: LEADER
I20260929 19:12:40.866798 23678 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52698ec68a4241f4933f13b4ebb7fe03" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } } peers { permanent_uuid: "949d544dddc640478e860fa73f0afbbc" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } } peers { permanent_uuid: "ef6365b7155549aabf35feef089324fc" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } }
I20260929 19:12:40.867268 23678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52698ec68a4241f4933f13b4ebb7fe03. Latest consensus state: current_term: 1 leader_uuid: "52698ec68a4241f4933f13b4ebb7fe03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52698ec68a4241f4933f13b4ebb7fe03" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } } peers { permanent_uuid: "949d544dddc640478e860fa73f0afbbc" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } } peers { permanent_uuid: "ef6365b7155549aabf35feef089324fc" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } } }
I20260929 19:12:40.867349 23678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [sys.catalog]: This master's current role is: LEADER
I20260929 19:12:40.867615 23728 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:12:40.867851 23728 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:12:40.870085 23644 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc [term 1 FOLLOWER]: Refusing update from remote peer 52698ec68a4241f4933f13b4ebb7fe03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:12:40.870357 23569 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc [term 1 FOLLOWER]: Refusing update from remote peer 52698ec68a4241f4933f13b4ebb7fe03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:12:40.870689 23678 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "949d544dddc640478e860fa73f0afbbc" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:12:40.870832 23678 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef6365b7155549aabf35feef089324fc" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:12:40.872429 23713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52698ec68a4241f4933f13b4ebb7fe03. Latest consensus state: current_term: 1 leader_uuid: "52698ec68a4241f4933f13b4ebb7fe03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52698ec68a4241f4933f13b4ebb7fe03" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } } peers { permanent_uuid: "949d544dddc640478e860fa73f0afbbc" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } } peers { permanent_uuid: "ef6365b7155549aabf35feef089324fc" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } } }
I20260929 19:12:40.872527 23713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:12:40.873610 23692 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52698ec68a4241f4933f13b4ebb7fe03. Latest consensus state: current_term: 1 leader_uuid: "52698ec68a4241f4933f13b4ebb7fe03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52698ec68a4241f4933f13b4ebb7fe03" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } } peers { permanent_uuid: "949d544dddc640478e860fa73f0afbbc" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } } peers { permanent_uuid: "ef6365b7155549aabf35feef089324fc" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } } }
I20260929 19:12:40.873736 23692 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:12:40.875170 23678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "52698ec68a4241f4933f13b4ebb7fe03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52698ec68a4241f4933f13b4ebb7fe03" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } } peers { permanent_uuid: "949d544dddc640478e860fa73f0afbbc" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } } peers { permanent_uuid: "ef6365b7155549aabf35feef089324fc" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } } }
I20260929 19:12:40.875252 23678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [sys.catalog]: This master's current role is: LEADER
I20260929 19:12:40.875654 23713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "52698ec68a4241f4933f13b4ebb7fe03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52698ec68a4241f4933f13b4ebb7fe03" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } } peers { permanent_uuid: "949d544dddc640478e860fa73f0afbbc" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } } peers { permanent_uuid: "ef6365b7155549aabf35feef089324fc" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } } }
I20260929 19:12:40.875735 23713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:12:40.875926 23729 mvcc.cc:205] Tried to move back new op lower bound from 7334744722921164800 to 7334744722911051776. Current Snapshot: MvccSnapshot[applied={T|T < 7334744722921164800}]
I20260929 19:12:40.876093 23730 mvcc.cc:205] Tried to move back new op lower bound from 7334744722921164800 to 7334744722911051776. Current Snapshot: MvccSnapshot[applied={T|T < 7334744722921164800}]
I20260929 19:12:40.876888 23678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "52698ec68a4241f4933f13b4ebb7fe03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52698ec68a4241f4933f13b4ebb7fe03" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } } peers { permanent_uuid: "949d544dddc640478e860fa73f0afbbc" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } } peers { permanent_uuid: "ef6365b7155549aabf35feef089324fc" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } } }
I20260929 19:12:40.876967 23678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03 [sys.catalog]: This master's current role is: LEADER
I20260929 19:12:40.877385 23692 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "52698ec68a4241f4933f13b4ebb7fe03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52698ec68a4241f4933f13b4ebb7fe03" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 39759 } } peers { permanent_uuid: "949d544dddc640478e860fa73f0afbbc" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36929 } } peers { permanent_uuid: "ef6365b7155549aabf35feef089324fc" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 45669 } } }
I20260929 19:12:40.877481 23692 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:12:40.877647 23732 mvcc.cc:205] Tried to move back new op lower bound from 7334744722921164800 to 7334744722911051776. Current Snapshot: MvccSnapshot[applied={T|T < 7334744722921164800}]
I20260929 19:12:40.879252 23728 catalog_manager.cc:1347] Generated new cluster ID: e721f737fd8445b3823aa66733e3fa74
I20260929 19:12:40.879312 23728 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:12:40.922001 23728 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:12:40.922775 23728 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:12:40.936061 23728 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52698ec68a4241f4933f13b4ebb7fe03: Generated new TSK 0
I20260929 19:12:40.936409 23728 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:12:41.767158 23703 catalog_manager.cc:1259] Loaded cluster ID: e721f737fd8445b3823aa66733e3fa74
I20260929 19:12:41.767256 23703 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc: loading cluster ID for follower catalog manager: success
I20260929 19:12:41.768231 23703 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc: acquiring CA information for follower catalog manager: success
I20260929 19:12:41.768564 23703 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ef6365b7155549aabf35feef089324fc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 19:12:41.812367 23724 catalog_manager.cc:1259] Loaded cluster ID: e721f737fd8445b3823aa66733e3fa74
I20260929 19:12:41.812474 23724 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc: loading cluster ID for follower catalog manager: success
I20260929 19:12:41.813437 23724 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc: acquiring CA information for follower catalog manager: success
I20260929 19:12:41.813721 23724 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 949d544dddc640478e860fa73f0afbbc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 19:12:41.989784 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 23452
I20260929 19:12:41.995151 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 23527
I20260929 19:12:41.999873 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 23602
2026-09-29T19:12:42Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2815 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:12:42 dist-test-slave-wftq krb5kdc[23743](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:12:42 dist-test-slave-wftq krb5kdc[23743](info): set up 2 sockets
Sep 29 19:12:42 dist-test-slave-wftq krb5kdc[23743](info): commencing operation
krb5kdc: starting...
W20260929 19:12:42.949303 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:12:43 dist-test-slave-wftq krb5kdc[23743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:12:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:12:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:12:43.072003 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:43807
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:43169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:43807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:12:43.151252 23756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:43.151461 23756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:43.151491 23756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:43.152629 23756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:12:43.152673 23756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:43.152690 23756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:43.152705 23756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:12:43.152719 23756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:12:43.154199 23756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:43169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:43807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:43807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:43.154443 23756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:43.154639 23756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:43.159847 23764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:43.159893 23761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:43.159847 23762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:43.160120 23756 server_base.cc:1034] running on GCE node
I20260929 19:12:43.160391 23756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:43.160758 23756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:43.161927 23756 hybrid_clock.cc:648] HybridClock initialized: now 1790709163161898 us; error 49 us; skew 500 ppm
Sep 29 19:12:43 dist-test-slave-wftq krb5kdc[23743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709163, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:43.164774 23756 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:12:43.165299 23756 webserver.cc:460] Webserver started at http://127.9.236.62:40295/ using document root <none> and password file <none>
I20260929 19:12:43.165552 23756 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:43.165602 23756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:12:43.165741 23756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:12:43.166937 23756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "087061d01acb47ffaffca4681d2770c7"
format_stamp: "Formatted at 2026-09-29 19:12:43 on dist-test-slave-wftq"
server_key: "c2d9e25ad353831f1bb7b6fe03e624ee"
server_key_iv: "90605d168370c683d1962506c6d73c61"
server_key_version: "encryptionkey@0"
I20260929 19:12:43.167313 23756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "087061d01acb47ffaffca4681d2770c7"
format_stamp: "Formatted at 2026-09-29 19:12:43 on dist-test-slave-wftq"
server_key: "c2d9e25ad353831f1bb7b6fe03e624ee"
server_key_iv: "90605d168370c683d1962506c6d73c61"
server_key_version: "encryptionkey@0"
I20260929 19:12:43.168710 23756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:12:43.169510 23771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:43.169737 23756 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 19:12:43.169832 23756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "087061d01acb47ffaffca4681d2770c7"
format_stamp: "Formatted at 2026-09-29 19:12:43 on dist-test-slave-wftq"
server_key: "c2d9e25ad353831f1bb7b6fe03e624ee"
server_key_iv: "90605d168370c683d1962506c6d73c61"
server_key_version: "encryptionkey@0"
I20260929 19:12:43.169893 23756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:43.178432 23756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:43.180303 23756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:43.180472 23756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:43.185029 23756 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:43807
I20260929 19:12:43.185081 23823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:43807 every 8 connection(s)
I20260929 19:12:43.185470 23756 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:12:43.186388 23824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:12:43.187366 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 23756
I20260929 19:12:43.187454 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:12:43.187705 10160 external_mini_cluster.cc:1428] Setting key e8f3c870f979a935319d9cd429cc0ec4
I20260929 19:12:43.189136 23824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7: Bootstrap starting.
I20260929 19:12:43.189749 23824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7: Neither blocks nor log segments found. Creating new log.
I20260929 19:12:43.190044 23824 log.cc:824] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7: Log is configured to *not* fsync() on all Append() calls
I20260929 19:12:43.190713 23824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7: No bootstrap required, opened a new log
I20260929 19:12:43.192853 23824 raft_consensus.cc:348] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087061d01acb47ffaffca4681d2770c7" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43807 } }
I20260929 19:12:43.193037 23824 raft_consensus.cc:374] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:12:43.193073 23824 raft_consensus.cc:729] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 087061d01acb47ffaffca4681d2770c7, State: Initialized, Role: FOLLOWER
I20260929 19:12:43.193312 23824 consensus_queue.cc:260] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087061d01acb47ffaffca4681d2770c7" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43807 } }
I20260929 19:12:43.193405 23829 raft_consensus.cc:482] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:12:43.193562 23829 raft_consensus.cc:504] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087061d01acb47ffaffca4681d2770c7" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43807 } }
I20260929 19:12:43.193691 23829 leader_election.cc:302] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 087061d01acb47ffaffca4681d2770c7; no voters:
I20260929 19:12:43.193874 23829 leader_election.cc:288] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260929 19:12:43.193953 23830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:12:43.194056 23830 raft_consensus.cc:482] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:12:43.194137 23830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:12:43.194842 23830 raft_consensus.cc:504] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087061d01acb47ffaffca4681d2770c7" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43807 } }
I20260929 19:12:43.194933 23830 leader_election.cc:302] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 087061d01acb47ffaffca4681d2770c7; no voters:
I20260929 19:12:43.195029 23830 leader_election.cc:288] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:12:43.195076 23830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [term 1 FOLLOWER]: Leader election won for term 1
Sep 29 19:12:43 dist-test-slave-wftq krb5kdc[23743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:12:43.195122 23830 raft_consensus.cc:686] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [term 1 LEADER]: Becoming Leader. State: Replica: 087061d01acb47ffaffca4681d2770c7, State: Running, Role: LEADER
I20260929 19:12:43.195271 23830 consensus_queue.cc:237] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087061d01acb47ffaffca4681d2770c7" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43807 } }
I20260929 19:12:43.195101 23824 raft_consensus.cc:388] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [term 1 LEADER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:12:43.195587 23829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 087061d01acb47ffaffca4681d2770c7. Latest consensus state: current_term: 1 leader_uuid: "087061d01acb47ffaffca4681d2770c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087061d01acb47ffaffca4681d2770c7" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43807 } } }
I20260929 19:12:43.195695 23829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [sys.catalog]: This master's current role is: LEADER
I20260929 19:12:43.195642 23824 raft_consensus.cc:465] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [term 1 LEADER]: Not starting leader election -- already a leader
I20260929 19:12:43.195854 23829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "087061d01acb47ffaffca4681d2770c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087061d01acb47ffaffca4681d2770c7" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43807 } } }
I20260929 19:12:43.195986 23829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [sys.catalog]: This master's current role is: LEADER
I20260929 19:12:43.196087 23832 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:12:43.195948 23824 sys_catalog.cc:564] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:12:43.196595 23832 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:12:43.198455 23832 catalog_manager.cc:1347] Generated new cluster ID: 8bd3327dbb3a49afbe8f33620de2f766
I20260929 19:12:43.198511 23832 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:12:43.199525 23827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:43.188268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56924 (local address 127.9.236.62:43807)
0929 19:12:43.188488 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:43.188499 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:43.188536 (+ 37us) server_negotiation.cc:407] Connection header received
0929 19:12:43.189050 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:43.189059 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:43.189142 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:43.189295 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:43.190076 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:43.191147 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:43.192308 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:43.192747 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:43.192832 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:43.195639 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:43.195645 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:43.195645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:43.195658 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:43.197646 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:43.198139 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:43.198144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:43.198144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:43.198250 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:43.198469 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:43.198470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:43.198471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:43.198793 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:43.198908 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:43.199150 (+ 242us) server_negotiation.cc:299] Negotiation successful
0929 19:12:43.199310 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":93,"threads_started":1}
I20260929 19:12:43.200776 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:12:43.211089 23832 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:12:43.211889 23832 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:12:43.219310 23832 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 087061d01acb47ffaffca4681d2770c7: Generated new TSK 0
I20260929 19:12:43.219553 23832 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:12:43.255080 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 23756
2026-09-29T19:12:43Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1253 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:12:43 dist-test-slave-wftq krb5kdc[23850](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:12:43 dist-test-slave-wftq krb5kdc[23850](info): set up 2 sockets
Sep 29 19:12:43 dist-test-slave-wftq krb5kdc[23850](info): commencing operation
krb5kdc: starting...
W20260929 19:12:44.205241 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:12:44 dist-test-slave-wftq krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:12:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:12:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:12:44.328882 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:46247
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:46237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:46247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:12:44.414770 23863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:44.414996 23863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:44.415026 23863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:44.416206 23863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:12:44.416246 23863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:44.416265 23863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:44.416280 23863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:12:44.416294 23863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:12:44.417852 23863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:46237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:46247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:46247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:44.418113 23863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:44.418346 23863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:44.423624 23869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:44.423628 23868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:44.423784 23863 server_base.cc:1034] running on GCE node
W20260929 19:12:44.423624 23871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:44.424252 23863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:44.424587 23863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:44.425737 23863 hybrid_clock.cc:648] HybridClock initialized: now 1790709164425718 us; error 36 us; skew 500 ppm
Sep 29 19:12:44 dist-test-slave-wftq krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709164, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:44.428171 23863 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:12:44.428625 23863 webserver.cc:460] Webserver started at http://127.9.236.62:36971/ using document root <none> and password file <none>
I20260929 19:12:44.428820 23863 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:44.428864 23863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:12:44.428963 23863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:12:44.430033 23863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "5e59deb6c22c4d20ba8546127ccb13fe"
format_stamp: "Formatted at 2026-09-29 19:12:44 on dist-test-slave-wftq"
server_key: "57b1faa641d7082f1e04a0d1af39986b"
server_key_iv: "ac3ad01b4db3de1e38e80148686e4d18"
server_key_version: "encryptionkey@0"
I20260929 19:12:44.430338 23863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "5e59deb6c22c4d20ba8546127ccb13fe"
format_stamp: "Formatted at 2026-09-29 19:12:44 on dist-test-slave-wftq"
server_key: "57b1faa641d7082f1e04a0d1af39986b"
server_key_iv: "ac3ad01b4db3de1e38e80148686e4d18"
server_key_version: "encryptionkey@0"
I20260929 19:12:44.431540 23863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:12:44.432220 23878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:44.432406 23863 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:44.432494 23863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "5e59deb6c22c4d20ba8546127ccb13fe"
format_stamp: "Formatted at 2026-09-29 19:12:44 on dist-test-slave-wftq"
server_key: "57b1faa641d7082f1e04a0d1af39986b"
server_key_iv: "ac3ad01b4db3de1e38e80148686e4d18"
server_key_version: "encryptionkey@0"
I20260929 19:12:44.432552 23863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:44.462952 23863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:44.464995 23863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:44.465164 23863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:44.469214 23863 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:46247
I20260929 19:12:44.469290 23930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:46247 every 8 connection(s)
I20260929 19:12:44.469596 23863 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:12:44.470553 23931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:12:44.473209 23931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe: Bootstrap starting.
I20260929 19:12:44.473829 23931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe: Neither blocks nor log segments found. Creating new log.
I20260929 19:12:44.474112 23931 log.cc:824] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe: Log is configured to *not* fsync() on all Append() calls
I20260929 19:12:44.474462 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 23863
I20260929 19:12:44.474550 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:12:44.474754 10160 external_mini_cluster.cc:1428] Setting key 7d9bd08c6bfd2205342e8afb8513b241
I20260929 19:12:44.474952 23931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe: No bootstrap required, opened a new log
I20260929 19:12:44.477586 23931 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46247 } }
I20260929 19:12:44.477730 23931 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:12:44.477752 23931 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e59deb6c22c4d20ba8546127ccb13fe, State: Initialized, Role: FOLLOWER
I20260929 19:12:44.477869 23931 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46247 } }
I20260929 19:12:44.477950 23931 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:12:44.477972 23931 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:12:44.478018 23931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:12:44.478602 23931 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46247 } }
I20260929 19:12:44.478694 23931 leader_election.cc:302] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e59deb6c22c4d20ba8546127ccb13fe; no voters:
I20260929 19:12:44.478863 23931 leader_election.cc:288] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:12:44.478899 23936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:12:44.479009 23936 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 1 LEADER]: Becoming Leader. State: Replica: 5e59deb6c22c4d20ba8546127ccb13fe, State: Running, Role: LEADER
I20260929 19:12:44.479148 23931 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:12:44.479157 23936 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46247 } }
I20260929 19:12:44.480278 23938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e59deb6c22c4d20ba8546127ccb13fe. Latest consensus state: current_term: 1 leader_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46247 } } }
I20260929 19:12:44.480377 23938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [sys.catalog]: This master's current role is: LEADER
I20260929 19:12:44.480419 23937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46247 } } }
I20260929 19:12:44.480551 23937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [sys.catalog]: This master's current role is: LEADER
I20260929 19:12:44.480753 23948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:12:44.481488 23948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 19:12:44 dist-test-slave-wftq krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:12:44.483474 23948 catalog_manager.cc:1347] Generated new cluster ID: 545d72676b5444cd87c4aae388c586d3
I20260929 19:12:44.483529 23948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:12:44.485805 23935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:44.475306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59570 (local address 127.9.236.62:46247)
0929 19:12:44.475519 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:44.475527 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:44.475552 (+ 25us) server_negotiation.cc:407] Connection header received
0929 19:12:44.476375 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:44.476382 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:44.476433 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:44.476556 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:44.477164 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:44.478197 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:44.478981 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:44.479558 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:44.479665 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:44.482569 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:44.482572 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:44.482573 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:44.482584 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:44.484373 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:44.484694 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:44.484697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:44.484697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:44.484785 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:44.484948 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:44.484949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:44.484950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:44.485249 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:44.485408 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:44.485485 (+ 77us) server_negotiation.cc:299] Negotiation successful
0929 19:12:44.485620 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":108,"threads_started":1}
I20260929 19:12:44.503068 23948 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:12:44.503727 23948 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:12:44.508939 23948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe: Generated new TSK 0
I20260929 19:12:44.509178 23948 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.236.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
I20260929 19:12:44.541779 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:46247
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:46237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:12:44.642102 23957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:44.642340 23957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:44.642369 23957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:44.643560 23957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:44.643599 23957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:44.643623 23957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:12:44.645066 23957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:46237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:46247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:44.645319 23957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:44.645529 23957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:44.650768 23962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:44.650830 23965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:44.650925 23957 server_base.cc:1034] running on GCE node
W20260929 19:12:44.650777 23963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:44.651271 23957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:44.651539 23957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:44.652695 23957 hybrid_clock.cc:648] HybridClock initialized: now 1790709164652676 us; error 33 us; skew 500 ppm
Sep 29 19:12:44 dist-test-slave-wftq krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709164, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:44.655440 23957 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:12:44.655925 23957 webserver.cc:460] Webserver started at http://127.9.236.1:41895/ using document root <none> and password file <none>
I20260929 19:12:44.656188 23957 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:44.656248 23957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:12:44.656381 23957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:12:44.657562 23957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "b0625621a9574adca65c1fa807eb83a8"
format_stamp: "Formatted at 2026-09-29 19:12:44 on dist-test-slave-wftq"
server_key: "52346534a14ae74473aa14585c72af58"
server_key_iv: "75e1a27b9083bf3701d9bc37530b68f6"
server_key_version: "encryptionkey@0"
I20260929 19:12:44.657900 23957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "b0625621a9574adca65c1fa807eb83a8"
format_stamp: "Formatted at 2026-09-29 19:12:44 on dist-test-slave-wftq"
server_key: "52346534a14ae74473aa14585c72af58"
server_key_iv: "75e1a27b9083bf3701d9bc37530b68f6"
server_key_version: "encryptionkey@0"
I20260929 19:12:44.659229 23957 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:44.660033 23972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:44.660212 23957 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:44.660301 23957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "b0625621a9574adca65c1fa807eb83a8"
format_stamp: "Formatted at 2026-09-29 19:12:44 on dist-test-slave-wftq"
server_key: "52346534a14ae74473aa14585c72af58"
server_key_iv: "75e1a27b9083bf3701d9bc37530b68f6"
server_key_version: "encryptionkey@0"
I20260929 19:12:44.660372 23957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:44.667695 23957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:44.669821 23957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:44.669960 23957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:44.670185 23957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:12:44.670506 23957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:12:44.670544 23957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:44.670576 23957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:12:44.670600 23957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:44.677412 23957 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:42881
I20260929 19:12:44.677488 24085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:42881 every 8 connection(s)
I20260929 19:12:44.677815 23957 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:12:44.678164 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 23957
I20260929 19:12:44.678269 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:12:44.678476 10160 external_mini_cluster.cc:1428] Setting key 781e4f1e8b60cd6e59803e7276588572
Sep 29 19:12:44 dist-test-slave-wftq krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709164, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.236.2@KRBTEST.COM; defaulting to no policy
I20260929 19:12:44.688473 23935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:44.678768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:60921 (local address 127.9.236.62:46247)
0929 19:12:44.678834 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:44.678836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:44.679095 (+ 259us) server_negotiation.cc:407] Connection header received
0929 19:12:44.679681 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:44.679682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:44.679696 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:44.679765 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:44.680836 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:44.681502 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:44.682916 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:44.683135 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:44.683224 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:44.685837 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:44.685838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:44.685839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:44.685847 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:44.687122 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:44.687511 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:44.687512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:44.687512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:44.687565 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:44.687799 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:44.687799 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:44.687800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:44.687935 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:44.687992 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:44.688279 (+ 287us) server_negotiation.cc:299] Negotiation successful
0929 19:12:44.688371 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 19:12:44.688671 24088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:44.678735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.1:60921)
0929 19:12:44.678943 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:44.678992 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:44.679565 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:44.679837 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:44.679841 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:44.680469 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:44.680731 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:44.680738 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:44.681603 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:44.681604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:44.682794 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:44.682797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:44.683243 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:44.683245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:44.683422 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:44.684002 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:44.684013 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:44.685740 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:44.687207 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:44.687216 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:44.687218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:44.687448 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:44.687617 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:44.687618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:44.687618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:44.687731 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:44.687992 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:44.688000 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:44.688161 (+ 161us) client_negotiation.cc:769] Sending connection context
0929 19:12:44.688254 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 19:12:44.688436 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":94,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.2@KRBTEST.COM" created.
I20260929 19:12:44.689314 24086 heartbeater.cc:347] Connected to a master server at 127.9.236.62:46247
I20260929 19:12:44.689415 24086 heartbeater.cc:464] Registering TS with master...
I20260929 19:12:44.689618 24086 heartbeater.cc:511] Master 127.9.236.62:46247 requested a full tablet report, sending...
I20260929 19:12:44.690219 23895 ts_manager.cc:194] Registered new tserver with Master: b0625621a9574adca65c1fa807eb83a8 (127.9.236.1:42881)
I20260929 19:12:44.690865 23895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:60921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
I20260929 19:12:44.704821 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:46247
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:46237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:12:44.786940 24091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:44.787149 24091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:44.787171 24091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:44.788333 24091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:44.788380 24091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:44.788405 24091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:12:44.789855 24091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:46237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:46247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:44.790067 24091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:44.790292 24091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:44.795158 24099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:44.795193 24097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:44.795208 24096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:44.795447 24091 server_base.cc:1034] running on GCE node
I20260929 19:12:44.795742 24091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:44.795962 24091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:44.797145 24091 hybrid_clock.cc:648] HybridClock initialized: now 1790709164797122 us; error 37 us; skew 500 ppm
Sep 29 19:12:44 dist-test-slave-wftq krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709164, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:44.799785 24091 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:12:44.800223 24091 webserver.cc:460] Webserver started at http://127.9.236.2:44725/ using document root <none> and password file <none>
I20260929 19:12:44.800424 24091 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:44.800469 24091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:12:44.800575 24091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:12:44.801640 24091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "ca5b39449c13474f98f601cb46c0ea0b"
format_stamp: "Formatted at 2026-09-29 19:12:44 on dist-test-slave-wftq"
server_key: "4a0ef27757fda696f662487d2de1ea23"
server_key_iv: "9bbbb99f824d9879e9f6649073d03e9b"
server_key_version: "encryptionkey@0"
I20260929 19:12:44.801962 24091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "ca5b39449c13474f98f601cb46c0ea0b"
format_stamp: "Formatted at 2026-09-29 19:12:44 on dist-test-slave-wftq"
server_key: "4a0ef27757fda696f662487d2de1ea23"
server_key_iv: "9bbbb99f824d9879e9f6649073d03e9b"
server_key_version: "encryptionkey@0"
I20260929 19:12:44.803186 24091 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:12:44.803889 24106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:44.804085 24091 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:12:44.804164 24091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "ca5b39449c13474f98f601cb46c0ea0b"
format_stamp: "Formatted at 2026-09-29 19:12:44 on dist-test-slave-wftq"
server_key: "4a0ef27757fda696f662487d2de1ea23"
server_key_iv: "9bbbb99f824d9879e9f6649073d03e9b"
server_key_version: "encryptionkey@0"
I20260929 19:12:44.804244 24091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:44.843528 24091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:44.845629 24091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:44.845762 24091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:44.845993 24091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:12:44.846302 24091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:12:44.846338 24091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:44.846365 24091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:12:44.846382 24091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:44.852604 24091 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:43261
I20260929 19:12:44.852756 24219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:43261 every 8 connection(s)
I20260929 19:12:44.852964 24091 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
Sep 29 19:12:44 dist-test-slave-wftq krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709164, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:12:44.860852 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 24091
I20260929 19:12:44.860952 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:12:44.861172 10160 external_mini_cluster.cc:1428] Setting key 6024d85d7dd78cbcdc48625707cbc009
I20260929 19:12:44.863231 23935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:44.853761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:45929 (local address 127.9.236.62:46247)
0929 19:12:44.853845 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:44.853848 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:44.854102 (+ 254us) server_negotiation.cc:407] Connection header received
0929 19:12:44.854749 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:44.854750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:44.854761 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:44.854819 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:44.855885 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:44.856411 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:44.857596 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:44.857784 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:44.857841 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:44.860149 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:44.860150 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:44.860150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:44.860157 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:44.861887 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:44.862336 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:44.862337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:44.862337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:44.862388 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:44.862603 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:44.862604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:44.862605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:44.862723 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:44.862783 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:44.863060 (+ 277us) server_negotiation.cc:299] Negotiation successful
0929 19:12:44.863135 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 19:12:44.863440 24222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:44.853683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.2:45929)
0929 19:12:44.853936 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:44.853994 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:44.854643 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:44.854887 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:44.854891 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:44.855543 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:44.855783 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:44.855788 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:44.856547 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:44.856548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:44.857516 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:44.857519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:44.857852 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:44.857853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:44.857962 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:44.858536 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:44.858545 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:44.860058 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:44.861954 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:44.861963 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:44.861965 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:44.862262 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:44.862442 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:44.862443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:44.862444 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:44.862549 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:44.862791 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:44.862798 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:44.862943 (+ 145us) client_negotiation.cc:769] Sending connection context
0929 19:12:44.863031 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 19:12:44.863220 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":86,"threads_started":1}
I20260929 19:12:44.864095 24220 heartbeater.cc:347] Connected to a master server at 127.9.236.62:46247
I20260929 19:12:44.864208 24220 heartbeater.cc:464] Registering TS with master...
I20260929 19:12:44.864391 24220 heartbeater.cc:511] Master 127.9.236.62:46247 requested a full tablet report, sending...
I20260929 19:12:44.864786 23895 ts_manager.cc:194] Registered new tserver with Master: ca5b39449c13474f98f601cb46c0ea0b (127.9.236.2:43261)
I20260929 19:12:44.865211 23895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:45929
WARNING: no policy specified for kudu/127.9.236.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
I20260929 19:12:44.886312 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:46247
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:46237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:12:44.972474 24225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:44.972710 24225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:44.972743 24225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:44.974023 24225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:44.974071 24225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:44.974097 24225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:12:44.975553 24225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:46237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:46247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:44.975755 24225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:44.975957 24225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:44.981416 24233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:44.981413 24230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:44.981786 24225 server_base.cc:1034] running on GCE node
W20260929 19:12:44.981442 24231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:44.982076 24225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:44.982362 24225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:44.983569 24225 hybrid_clock.cc:648] HybridClock initialized: now 1790709164983542 us; error 44 us; skew 500 ppm
Sep 29 19:12:44 dist-test-slave-wftq krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709164, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:44.986312 24225 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:12:44.986837 24225 webserver.cc:460] Webserver started at http://127.9.236.3:42143/ using document root <none> and password file <none>
I20260929 19:12:44.987068 24225 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:44.987118 24225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:12:44.987250 24225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:12:44.988436 24225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "90886c62688a4d4c92ebf9f840f0b5d2"
format_stamp: "Formatted at 2026-09-29 19:12:44 on dist-test-slave-wftq"
server_key: "1d890a1c5f8135852767dbfcfb5238dc"
server_key_iv: "2f25a1e23de69651bdc13cc0b0ffcf95"
server_key_version: "encryptionkey@0"
I20260929 19:12:44.988793 24225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "90886c62688a4d4c92ebf9f840f0b5d2"
format_stamp: "Formatted at 2026-09-29 19:12:44 on dist-test-slave-wftq"
server_key: "1d890a1c5f8135852767dbfcfb5238dc"
server_key_iv: "2f25a1e23de69651bdc13cc0b0ffcf95"
server_key_version: "encryptionkey@0"
I20260929 19:12:44.990239 24225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:12:44.991118 24240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:44.991324 24225 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:44.991423 24225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "90886c62688a4d4c92ebf9f840f0b5d2"
format_stamp: "Formatted at 2026-09-29 19:12:44 on dist-test-slave-wftq"
server_key: "1d890a1c5f8135852767dbfcfb5238dc"
server_key_iv: "2f25a1e23de69651bdc13cc0b0ffcf95"
server_key_version: "encryptionkey@0"
I20260929 19:12:44.991518 24225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:45.018321 24225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:45.020619 24225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:45.020772 24225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:45.021030 24225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:12:45.021425 24225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:12:45.021476 24225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:45.021513 24225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:12:45.021543 24225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:45.029187 24225 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:40443
I20260929 19:12:45.029241 24353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:40443 every 8 connection(s)
I20260929 19:12:45.029641 24225 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:12:45.032029 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 24225
I20260929 19:12:45.032117 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:12:45.032297 10160 external_mini_cluster.cc:1428] Setting key 37a3203675ab1faf0d4df1d6d17812f6
Sep 29 19:12:45 dist-test-slave-wftq krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709164, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:12:45.039395 23935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.030489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:52425 (local address 127.9.236.62:46247)
0929 19:12:45.030574 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:45.030577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:45.030749 (+ 172us) server_negotiation.cc:407] Connection header received
0929 19:12:45.031272 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:45.031273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:45.031286 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:45.031375 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:45.032429 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.033061 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.034176 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.034385 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.034441 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.036732 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:45.036733 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:45.036733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:45.036739 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:45.038046 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:45.038529 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:45.038529 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:45.038530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:45.038576 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:45.038843 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:45.038844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:45.038844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:45.038935 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:45.038984 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:45.039260 (+ 276us) server_negotiation.cc:299] Negotiation successful
0929 19:12:45.039327 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 19:12:45.039695 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.030423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.3:52425)
0929 19:12:45.030615 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:45.030657 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:45.031180 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:45.031443 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:45.031446 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:45.032079 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:45.032325 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.032331 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.033238 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.033239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.034099 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.034102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.034451 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.034453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.034615 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.035139 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:45.035147 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:45.036630 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:45.038149 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:45.038157 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:45.038159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:45.038451 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:45.038659 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:45.038660 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:45.038660 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:45.038780 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:45.038988 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:45.038992 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:45.039136 (+ 144us) client_negotiation.cc:769] Sending connection context
0929 19:12:45.039243 (+ 107us) client_negotiation.cc:240] Negotiation successful
0929 19:12:45.039394 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":112,"threads_started":1}
I20260929 19:12:45.040231 24354 heartbeater.cc:347] Connected to a master server at 127.9.236.62:46247
I20260929 19:12:45.040339 24354 heartbeater.cc:464] Registering TS with master...
I20260929 19:12:45.040517 24354 heartbeater.cc:511] Master 127.9.236.62:46247 requested a full tablet report, sending...
I20260929 19:12:45.040984 23895 ts_manager.cc:194] Registered new tserver with Master: 90886c62688a4d4c92ebf9f840f0b5d2 (127.9.236.3:40443)
I20260929 19:12:45.041404 23895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:52425
I20260929 19:12:45.044562 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:12:45.052616 23935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.046264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40990 (local address 127.9.236.62:46247)
0929 19:12:45.046341 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:45.046343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:45.046499 (+ 156us) server_negotiation.cc:407] Connection header received
0929 19:12:45.046523 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:45.046524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:45.046543 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:45.046596 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:45.047476 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.047988 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.048746 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.048935 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.048996 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.050115 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:45.050116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:45.050117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:45.050123 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:45.051492 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:45.051790 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:45.051790 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:45.051791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:45.051837 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:45.052042 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:45.052043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:45.052043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:45.052192 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:45.052274 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:45.052454 (+ 180us) server_negotiation.cc:299] Negotiation successful
0929 19:12:45.052536 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 19:12:45.054397 23895 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40990:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:12:45.055968 23895 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:12:45.062597 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.058892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54072 (local address 127.9.236.2:43261)
0929 19:12:45.059066 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:45.059070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:45.059205 (+ 135us) server_negotiation.cc:407] Connection header received
0929 19:12:45.059244 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:45.059246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:45.059262 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:45.059337 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:12:45.059525 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.060565 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.061730 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.062209 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.062273 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.062333 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:45.062471 (+ 138us) server_negotiation.cc:299] Negotiation successful
0929 19:12:45.062511 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":84,"threads_started":1}
I20260929 19:12:45.062587 24368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.058665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:43261 (local address 127.0.0.1:54072)
0929 19:12:45.059104 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:45.059113 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:45.059159 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:45.059409 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:45.059411 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:45.059414 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:12:45.059459 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.059461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.060659 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.060660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.061658 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.061660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.062295 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.062296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.062374 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.062389 (+ 15us) client_negotiation.cc:769] Sending connection context
0929 19:12:45.062451 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 19:12:45.062511 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":72,"threads_started":1}
I20260929 19:12:45.063014 24365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.058467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:40443 (local address 127.0.0.1:51456)
0929 19:12:45.058642 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:45.058681 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:45.058742 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:45.059143 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:45.059146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:45.059148 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:12:45.059230 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.059233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.060616 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.060617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.061771 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.061773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.062737 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.062738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.062807 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.062810 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:12:45.062852 (+ 42us) client_negotiation.cc:240] Negotiation successful
0929 19:12:45.062904 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":2,"thread_start_us":74,"threads_started":1}
I20260929 19:12:45.063232 24369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.058707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33106 (local address 127.9.236.1:42881)
0929 19:12:45.058878 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:45.058884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:45.058926 (+ 42us) server_negotiation.cc:407] Connection header received
0929 19:12:45.059022 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:45.059025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:45.059039 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:45.059121 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:12:45.059377 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.060159 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.061758 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.062521 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.062595 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.062660 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:45.062943 (+ 283us) server_negotiation.cc:299] Negotiation successful
0929 19:12:45.063051 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260929 19:12:45.063309 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.058570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51456 (local address 127.9.236.3:40443)
0929 19:12:45.058911 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:45.058915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:45.058925 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:12:45.058970 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:45.058972 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:45.058992 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:45.059122 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:12:45.059336 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.060501 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.061868 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.062600 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.062659 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.062707 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:45.063164 (+ 457us) server_negotiation.cc:299] Negotiation successful
0929 19:12:45.063229 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":121,"threads_started":1}
I20260929 19:12:45.063827 24154 tablet_service.cc:1467] Processing CreateTablet for tablet 96bd7792985e4736b98fa9340fe365ee (DEFAULT_TABLE table=test-table [id=9b84825f0721436989fedc409ae8a623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:12:45.064239 24154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96bd7792985e4736b98fa9340fe365ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:12:45.064136 24288 tablet_service.cc:1467] Processing CreateTablet for tablet 96bd7792985e4736b98fa9340fe365ee (DEFAULT_TABLE table=test-table [id=9b84825f0721436989fedc409ae8a623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:12:45.064572 24288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96bd7792985e4736b98fa9340fe365ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:12:45.063017 24366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.058467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:42881 (local address 127.0.0.1:33106)
0929 19:12:45.058864 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:45.058873 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:45.058927 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:45.059231 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:45.059233 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:45.059236 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:12:45.059302 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.059304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.060262 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.060263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.061651 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.061654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.062700 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.062701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.062770 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.062775 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:12:45.062843 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 19:12:45.062943 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":27,"thread_start_us":70,"threads_started":1}
I20260929 19:12:45.065938 24020 tablet_service.cc:1467] Processing CreateTablet for tablet 96bd7792985e4736b98fa9340fe365ee (DEFAULT_TABLE table=test-table [id=9b84825f0721436989fedc409ae8a623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:12:45.066311 24020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96bd7792985e4736b98fa9340fe365ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:12:45.067447 24371 tablet_bootstrap.cc:492] T 96bd7792985e4736b98fa9340fe365ee P 90886c62688a4d4c92ebf9f840f0b5d2: Bootstrap starting.
I20260929 19:12:45.067873 24372 tablet_bootstrap.cc:492] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b: Bootstrap starting.
I20260929 19:12:45.068130 24371 tablet_bootstrap.cc:654] T 96bd7792985e4736b98fa9340fe365ee P 90886c62688a4d4c92ebf9f840f0b5d2: Neither blocks nor log segments found. Creating new log.
I20260929 19:12:45.068564 24372 tablet_bootstrap.cc:654] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b: Neither blocks nor log segments found. Creating new log.
I20260929 19:12:45.068665 24371 log.cc:824] T 96bd7792985e4736b98fa9340fe365ee P 90886c62688a4d4c92ebf9f840f0b5d2: Log is configured to *not* fsync() on all Append() calls
I20260929 19:12:45.069036 24372 log.cc:824] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b: Log is configured to *not* fsync() on all Append() calls
I20260929 19:12:45.069566 24375 tablet_bootstrap.cc:492] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8: Bootstrap starting.
I20260929 19:12:45.069612 24371 tablet_bootstrap.cc:492] T 96bd7792985e4736b98fa9340fe365ee P 90886c62688a4d4c92ebf9f840f0b5d2: No bootstrap required, opened a new log
I20260929 19:12:45.069687 24371 ts_tablet_manager.cc:1397] T 96bd7792985e4736b98fa9340fe365ee P 90886c62688a4d4c92ebf9f840f0b5d2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:12:45.069782 24372 tablet_bootstrap.cc:492] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b: No bootstrap required, opened a new log
I20260929 19:12:45.069844 24372 ts_tablet_manager.cc:1397] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:12:45.070632 24375 tablet_bootstrap.cc:654] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8: Neither blocks nor log segments found. Creating new log.
I20260929 19:12:45.071177 24375 log.cc:824] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8: Log is configured to *not* fsync() on all Append() calls
I20260929 19:12:45.071296 24371 raft_consensus.cc:348] T 96bd7792985e4736b98fa9340fe365ee P 90886c62688a4d4c92ebf9f840f0b5d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } } peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } }
I20260929 19:12:45.071452 24371 raft_consensus.cc:374] T 96bd7792985e4736b98fa9340fe365ee P 90886c62688a4d4c92ebf9f840f0b5d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:12:45.071488 24371 raft_consensus.cc:729] T 96bd7792985e4736b98fa9340fe365ee P 90886c62688a4d4c92ebf9f840f0b5d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90886c62688a4d4c92ebf9f840f0b5d2, State: Initialized, Role: FOLLOWER
I20260929 19:12:45.071620 24371 consensus_queue.cc:260] T 96bd7792985e4736b98fa9340fe365ee P 90886c62688a4d4c92ebf9f840f0b5d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } } peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } }
I20260929 19:12:45.071938 24371 ts_tablet_manager.cc:1428] T 96bd7792985e4736b98fa9340fe365ee P 90886c62688a4d4c92ebf9f840f0b5d2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 19:12:45.071959 24375 tablet_bootstrap.cc:492] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8: No bootstrap required, opened a new log
I20260929 19:12:45.071962 24354 heartbeater.cc:503] Master 127.9.236.62:46247 was elected leader, sending a full tablet report...
I20260929 19:12:45.072016 24375 ts_tablet_manager.cc:1397] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 19:12:45.071887 24372 raft_consensus.cc:348] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } } peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } }
I20260929 19:12:45.072047 24372 raft_consensus.cc:374] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:12:45.072081 24372 raft_consensus.cc:729] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5b39449c13474f98f601cb46c0ea0b, State: Initialized, Role: FOLLOWER
I20260929 19:12:45.072202 24372 consensus_queue.cc:260] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } } peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } }
I20260929 19:12:45.072453 24372 ts_tablet_manager.cc:1428] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260929 19:12:45.072538 24220 heartbeater.cc:503] Master 127.9.236.62:46247 was elected leader, sending a full tablet report...
I20260929 19:12:45.073900 24375 raft_consensus.cc:348] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } } peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } }
I20260929 19:12:45.074044 24375 raft_consensus.cc:374] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:12:45.074065 24375 raft_consensus.cc:729] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0625621a9574adca65c1fa807eb83a8, State: Initialized, Role: FOLLOWER
I20260929 19:12:45.074157 24375 consensus_queue.cc:260] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } } peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } }
I20260929 19:12:45.074450 24375 ts_tablet_manager.cc:1428] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 19:12:45.074555 24086 heartbeater.cc:503] Master 127.9.236.62:46247 was elected leader, sending a full tablet report...
I20260929 19:12:45.081472 24379 raft_consensus.cc:482] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:12:45.081566 24379 raft_consensus.cc:504] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } } peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } }
I20260929 19:12:45.081894 24379 leader_election.cc:288] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90886c62688a4d4c92ebf9f840f0b5d2 (127.9.236.3:40443), ca5b39449c13474f98f601cb46c0ea0b (127.9.236.2:43261)
I20260929 19:12:45.085907 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.082319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:47417 (local address 127.9.236.2:43261)
0929 19:12:45.082358 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:45.082360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:45.082439 (+ 79us) server_negotiation.cc:407] Connection header received
0929 19:12:45.082498 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:45.082499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:45.082506 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:45.082548 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:12:45.082689 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.083373 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.084956 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.085536 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.085617 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.085627 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:45.085807 (+ 180us) server_negotiation.cc:299] Negotiation successful
0929 19:12:45.085847 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:12:45.085933 24088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.082086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:40443 (local address 127.9.236.1:42983)
0929 19:12:45.082149 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:45.082163 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:45.082215 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:45.082383 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:45.082384 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:45.082387 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:12:45.082536 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.082538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.083510 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.083511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.084865 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.084868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.085619 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.085619 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.085727 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.085732 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:12:45.085791 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 19:12:45.085835 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":34}
I20260929 19:12:45.085940 24380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.082204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:43261 (local address 127.9.236.1:47417)
0929 19:12:45.082369 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:45.082382 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:45.082447 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:45.082555 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:45.082556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:45.082557 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:12:45.082630 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.082631 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.083484 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.083486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.084865 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.084868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.085619 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.085620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.085727 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.085732 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:12:45.085787 (+ 55us) client_negotiation.cc:240] Negotiation successful
0929 19:12:45.085834 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":85,"thread_start_us":107,"threads_started":1}
I20260929 19:12:45.086313 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.082164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:42983 (local address 127.9.236.3:40443)
0929 19:12:45.082214 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:45.082216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:45.082224 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:12:45.082284 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:45.082285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:45.082296 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:45.082343 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:12:45.082607 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.083378 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.084955 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.085550 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.085617 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.085627 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:45.086193 (+ 566us) server_negotiation.cc:299] Negotiation successful
0929 19:12:45.086261 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 19:12:45.086354 24174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96bd7792985e4736b98fa9340fe365ee" candidate_uuid: "b0625621a9574adca65c1fa807eb83a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5b39449c13474f98f601cb46c0ea0b" is_pre_election: true
I20260929 19:12:45.086520 24174 raft_consensus.cc:2454] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0625621a9574adca65c1fa807eb83a8 in term 0.
I20260929 19:12:45.086581 24308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96bd7792985e4736b98fa9340fe365ee" candidate_uuid: "b0625621a9574adca65c1fa807eb83a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" is_pre_election: true
I20260929 19:12:45.086746 24308 raft_consensus.cc:2454] T 96bd7792985e4736b98fa9340fe365ee P 90886c62688a4d4c92ebf9f840f0b5d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0625621a9574adca65c1fa807eb83a8 in term 0.
I20260929 19:12:45.086748 23973 leader_election.cc:302] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0625621a9574adca65c1fa807eb83a8, ca5b39449c13474f98f601cb46c0ea0b; no voters:
I20260929 19:12:45.086874 24379 raft_consensus.cc:2781] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:12:45.086939 24379 raft_consensus.cc:482] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:12:45.086993 24379 raft_consensus.cc:3037] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:12:45.087766 24379 raft_consensus.cc:504] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } } peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } }
I20260929 19:12:45.087886 24379 leader_election.cc:288] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 1 election: Requested vote from peers 90886c62688a4d4c92ebf9f840f0b5d2 (127.9.236.3:40443), ca5b39449c13474f98f601cb46c0ea0b (127.9.236.2:43261)
I20260929 19:12:45.088076 24308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96bd7792985e4736b98fa9340fe365ee" candidate_uuid: "b0625621a9574adca65c1fa807eb83a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90886c62688a4d4c92ebf9f840f0b5d2"
I20260929 19:12:45.088079 24174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96bd7792985e4736b98fa9340fe365ee" candidate_uuid: "b0625621a9574adca65c1fa807eb83a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5b39449c13474f98f601cb46c0ea0b"
I20260929 19:12:45.088147 24308 raft_consensus.cc:3037] T 96bd7792985e4736b98fa9340fe365ee P 90886c62688a4d4c92ebf9f840f0b5d2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:12:45.088148 24174 raft_consensus.cc:3037] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:12:45.088902 24308 raft_consensus.cc:2454] T 96bd7792985e4736b98fa9340fe365ee P 90886c62688a4d4c92ebf9f840f0b5d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0625621a9574adca65c1fa807eb83a8 in term 1.
I20260929 19:12:45.088918 24174 raft_consensus.cc:2454] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0625621a9574adca65c1fa807eb83a8 in term 1.
I20260929 19:12:45.089051 23973 leader_election.cc:302] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0625621a9574adca65c1fa807eb83a8, ca5b39449c13474f98f601cb46c0ea0b; no voters:
I20260929 19:12:45.089190 24379 raft_consensus.cc:2781] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:12:45.089366 24379 raft_consensus.cc:686] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [term 1 LEADER]: Becoming Leader. State: Replica: b0625621a9574adca65c1fa807eb83a8, State: Running, Role: LEADER
I20260929 19:12:45.089454 24379 consensus_queue.cc:237] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } } peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } }
I20260929 19:12:45.090247 23894 catalog_manager.cc:5495] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0625621a9574adca65c1fa807eb83a8 (127.9.236.1). New cstate: current_term: 1 leader_uuid: "b0625621a9574adca65c1fa807eb83a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } health_report { overall_health: UNKNOWN } } }
I20260929 19:12:45.098045 24369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.095672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33120 (local address 127.9.236.1:42881)
0929 19:12:45.095717 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:45.095719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:45.095728 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:12:45.095794 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:45.095794 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:45.095811 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:45.095864 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:12:45.095975 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.096443 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.097355 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.097543 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.097600 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.097773 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:12:45.097870 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:12:45.097927 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:45.097948 (+ 21us) server_negotiation.cc:299] Negotiation successful
0929 19:12:45.097965 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:12:45.100101 24174 raft_consensus.cc:1261] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b [term 1 FOLLOWER]: Refusing update from remote peer b0625621a9574adca65c1fa807eb83a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:12:45.100101 24308 raft_consensus.cc:1261] T 96bd7792985e4736b98fa9340fe365ee P 90886c62688a4d4c92ebf9f840f0b5d2 [term 1 FOLLOWER]: Refusing update from remote peer b0625621a9574adca65c1fa807eb83a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:12:45.100529 24379 consensus_queue.cc:1035] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:12:45.100713 24381 consensus_queue.cc:1035] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 19:12:45.103502 24221 tablet.cc:2369] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:12:45.104076 24386 mvcc.cc:205] Tried to move back new op lower bound from 7334744740246794240 to 7334744740207144960. Current Snapshot: MvccSnapshot[applied={T|T < 7334744740246794240}]
I20260929 19:12:45.106240 23894 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:12:45.106355 23894 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:12:45.107139 23894 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe: Sending DeleteTablet for 3 replicas of tablet 96bd7792985e4736b98fa9340fe365ee
I20260929 19:12:45.107476 24020 tablet_service.cc:1514] Processing DeleteTablet for tablet 96bd7792985e4736b98fa9340fe365ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:12:45 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:33106
I20260929 19:12:45.107546 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 23957
I20260929 19:12:45.108186 24288 tablet_service.cc:1514] Processing DeleteTablet for tablet 96bd7792985e4736b98fa9340fe365ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:12:45 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:51456
I20260929 19:12:45.108428 24394 tablet_replica.cc:330] T 96bd7792985e4736b98fa9340fe365ee P 90886c62688a4d4c92ebf9f840f0b5d2: stopping tablet replica
I20260929 19:12:45.108472 24154 tablet_service.cc:1514] Processing DeleteTablet for tablet 96bd7792985e4736b98fa9340fe365ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:12:45 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:54072
I20260929 19:12:45.108544 24394 raft_consensus.cc:2229] T 96bd7792985e4736b98fa9340fe365ee P 90886c62688a4d4c92ebf9f840f0b5d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:12:45.108647 24394 raft_consensus.cc:2258] T 96bd7792985e4736b98fa9340fe365ee P 90886c62688a4d4c92ebf9f840f0b5d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:12:45.109279 24394 ts_tablet_manager.cc:1905] T 96bd7792985e4736b98fa9340fe365ee P 90886c62688a4d4c92ebf9f840f0b5d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:12:45.111132 24394 ts_tablet_manager.cc:1918] T 96bd7792985e4736b98fa9340fe365ee P 90886c62688a4d4c92ebf9f840f0b5d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:12:45.111207 24394 log.cc:1196] T 96bd7792985e4736b98fa9340fe365ee P 90886c62688a4d4c92ebf9f840f0b5d2: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/96bd7792985e4736b98fa9340fe365ee
I20260929 19:12:45.111438 24394 ts_tablet_manager.cc:1939] T 96bd7792985e4736b98fa9340fe365ee P 90886c62688a4d4c92ebf9f840f0b5d2: Deleting consensus metadata
I20260929 19:12:45.111852 23879 catalog_manager.cc:4841] TS 90886c62688a4d4c92ebf9f840f0b5d2 (127.9.236.3:40443): tablet 96bd7792985e4736b98fa9340fe365ee (table test-table [id=9b84825f0721436989fedc409ae8a623]) successfully deleted
W20260929 19:12:45.113113 23881 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.236.1:42881 (error 108)
I20260929 19:12:45.113395 24395 tablet_replica.cc:330] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b: stopping tablet replica
I20260929 19:12:45.113483 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 24091
I20260929 19:12:45.113517 24395 raft_consensus.cc:2229] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:12:45.113577 24366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.113469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:42881 (local address 127.0.0.1:33126)
0929 19:12:45.113514 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:45.113545 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.1:42881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260929 19:12:45.113631 24395 raft_consensus.cc:2258] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260929 19:12:45.113667 23881 catalog_manager.cc:4584] TS b0625621a9574adca65c1fa807eb83a8 (127.9.236.1:42881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 96bd7792985e4736b98fa9340fe365ee: Network error: Client connection negotiation failed: client connection to 127.9.236.1:42881: connect: Connection refused (error 111)
I20260929 19:12:45.119071 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 24225
I20260929 19:12:45.119109 24366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.118986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:43261 (local address 127.0.0.1:54084)
0929 19:12:45.119045 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:45.119083 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.2:43261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260929 19:12:45.123777 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 23863
I20260929 19:12:45.128962 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:46247
--webserver_interface=127.9.236.62
--webserver_port=36971
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:46237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:46247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:12:45.215178 24397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:45.215427 24397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:45.215459 24397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:45.216653 24397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:12:45.216694 24397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:45.216712 24397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:45.216728 24397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:12:45.216743 24397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:12:45.218272 24397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:46237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:46247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:46247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=36971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:45.218530 24397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:45.218746 24397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:45.227793 24406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:45.227813 24404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:45.227813 24403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:45.227926 24397 server_base.cc:1034] running on GCE node
I20260929 19:12:45.228469 24397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:45.228816 24397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:45.229965 24397 hybrid_clock.cc:648] HybridClock initialized: now 1790709165229949 us; error 36 us; skew 500 ppm
Sep 29 19:12:45 dist-test-slave-wftq krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709165, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:45.232547 24397 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:12:45.233012 24397 webserver.cc:460] Webserver started at http://127.9.236.62:36971/ using document root <none> and password file <none>
I20260929 19:12:45.233263 24397 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:45.233332 24397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:12:45.234850 24397 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:45.235569 24413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:45.235765 24397 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:45.235858 24397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "5e59deb6c22c4d20ba8546127ccb13fe"
format_stamp: "Formatted at 2026-09-29 19:12:44 on dist-test-slave-wftq"
server_key: "57b1faa641d7082f1e04a0d1af39986b"
server_key_iv: "ac3ad01b4db3de1e38e80148686e4d18"
server_key_version: "encryptionkey@0"
I20260929 19:12:45.236258 24397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:45.259203 24397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:45.261148 24397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:45.261301 24397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:45.265568 24397 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:46247
I20260929 19:12:45.265643 24465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:46247 every 8 connection(s)
I20260929 19:12:45.265991 24397 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:12:45.267882 24466 sys_catalog.cc:263] Verifying existing consensus state
I20260929 19:12:45.268642 24466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe: Bootstrap starting.
I20260929 19:12:45.274534 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 24397
I20260929 19:12:45.274801 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:42881
--local_ip_for_outbound_sockets=127.9.236.1
--tserver_master_addrs=127.9.236.62:46247
--webserver_port=41895
--webserver_interface=127.9.236.1
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:46237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:12:45.279050 24466 log.cc:824] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe: Log is configured to *not* fsync() on all Append() calls
I20260929 19:12:45.281371 24466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:12:45.281708 24466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe: Bootstrap complete.
I20260929 19:12:45.284070 24466 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46247 } }
I20260929 19:12:45.284345 24466 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e59deb6c22c4d20ba8546127ccb13fe, State: Initialized, Role: FOLLOWER
I20260929 19:12:45.284520 24466 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46247 } }
I20260929 19:12:45.284596 24466 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:12:45.284644 24466 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:12:45.284696 24466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 1 FOLLOWER]: Advancing to term 2
I20260929 19:12:45.285849 24466 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46247 } }
I20260929 19:12:45.285954 24466 leader_election.cc:302] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e59deb6c22c4d20ba8546127ccb13fe; no voters:
I20260929 19:12:45.286134 24466 leader_election.cc:288] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 19:12:45.286182 24470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 2 FOLLOWER]: Leader election won for term 2
I20260929 19:12:45.286360 24470 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 2 LEADER]: Becoming Leader. State: Replica: 5e59deb6c22c4d20ba8546127ccb13fe, State: Running, Role: LEADER
I20260929 19:12:45.286451 24466 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:12:45.286598 24470 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46247 } }
I20260929 19:12:45.286984 24471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46247 } } }
I20260929 19:12:45.287077 24471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [sys.catalog]: This master's current role is: LEADER
I20260929 19:12:45.287432 24477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:12:45.287650 24472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e59deb6c22c4d20ba8546127ccb13fe. Latest consensus state: current_term: 2 leader_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46247 } } }
I20260929 19:12:45.287731 24472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [sys.catalog]: This master's current role is: LEADER
I20260929 19:12:45.288352 24477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:12:45.288516 24477 catalog_manager.cc:1259] Loaded cluster ID: 545d72676b5444cd87c4aae388c586d3
I20260929 19:12:45.288836 24477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:12:45.289165 24477 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260929 19:12:45.294204 24477 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260929 19:12:45.297008 24477 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:12:45.297291 24477 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe: Loaded TSK: 0
I20260929 19:12:45.297528 24477 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 19:12:45.364153 24468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:45.364367 24468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:45.364395 24468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:45.365571 24468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:45.365612 24468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:45.365645 24468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:12:45.367041 24468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:46237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:42881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=41895
--tserver_master_addrs=127.9.236.62:46247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:45.367255 24468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:45.367444 24468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:45.372380 24494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:45.372524 24468 server_base.cc:1034] running on GCE node
W20260929 19:12:45.372377 24493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:45.372380 24496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:45.373052 24468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:45.373411 24468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:45.374598 24468 hybrid_clock.cc:648] HybridClock initialized: now 1790709165374584 us; error 27 us; skew 500 ppm
Sep 29 19:12:45 dist-test-slave-wftq krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709165, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:45.377879 24468 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:12:45.378507 24468 webserver.cc:460] Webserver started at http://127.9.236.1:41895/ using document root <none> and password file <none>
I20260929 19:12:45.378769 24468 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:45.378849 24468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:12:45.380859 24468 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:12:45.381871 24503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:45.382099 24468 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:45.382210 24468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "b0625621a9574adca65c1fa807eb83a8"
format_stamp: "Formatted at 2026-09-29 19:12:44 on dist-test-slave-wftq"
server_key: "52346534a14ae74473aa14585c72af58"
server_key_iv: "75e1a27b9083bf3701d9bc37530b68f6"
server_key_version: "encryptionkey@0"
I20260929 19:12:45.382622 24468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:45.396502 24468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:45.398758 24468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:45.398869 24468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:45.399107 24468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:12:45.399587 24510 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 19:12:45.401878 24468 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 19:12:45.401942 24468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260929 19:12:45.401975 24468 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 19:12:45.402709 24468 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 19:12:45.402741 24468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260929 19:12:45.402853 24510 tablet_bootstrap.cc:492] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8: Bootstrap starting.
I20260929 19:12:45.408754 24468 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:42881
I20260929 19:12:45.408900 24617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:42881 every 8 connection(s)
I20260929 19:12:45.409220 24468 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:12:45.410291 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 24468
I20260929 19:12:45.410557 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:43261
--local_ip_for_outbound_sockets=127.9.236.2
--tserver_master_addrs=127.9.236.62:46247
--webserver_port=44725
--webserver_interface=127.9.236.2
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:46237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:12:45.415977 24510 log.cc:824] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8: Log is configured to *not* fsync() on all Append() calls
Sep 29 19:12:45 dist-test-slave-wftq krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709165, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:12:45.419344 24510 tablet_bootstrap.cc:492] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:12:45.419727 24510 tablet_bootstrap.cc:492] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8: Bootstrap complete.
I20260929 19:12:45.421023 24510 ts_tablet_manager.cc:1397] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.008s
I20260929 19:12:45.422689 24510 raft_consensus.cc:348] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } } peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } }
I20260929 19:12:45.423193 24510 raft_consensus.cc:729] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0625621a9574adca65c1fa807eb83a8, State: Initialized, Role: FOLLOWER
I20260929 19:12:45.423430 24510 consensus_queue.cc:260] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } } peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } }
I20260929 19:12:45.423822 24510 ts_tablet_manager.cc:1428] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260929 19:12:45.424274 24621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.409937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:51143 (local address 127.9.236.62:46247)
0929 19:12:45.411973 (+ 2036us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:45.411982 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:45.412029 (+ 47us) server_negotiation.cc:407] Connection header received
0929 19:12:45.412176 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:45.412181 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:45.412232 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:45.412359 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:45.413092 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.414029 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.414968 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.415420 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.415514 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.420024 (+ 4510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:45.420028 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:45.420029 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:45.420039 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:45.422083 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:45.422637 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:45.422640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:45.422640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:45.422748 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:45.423036 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:45.423037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:45.423038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:45.423326 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:45.423384 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:45.423765 (+ 381us) server_negotiation.cc:299] Negotiation successful
0929 19:12:45.423904 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1972,"thread_start_us":128,"threads_started":1}
I20260929 19:12:45.424726 24620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.409823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.1:51143)
0929 19:12:45.409979 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:45.410018 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:45.410677 (+ 659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:45.412503 (+ 1826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:45.412507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:45.412803 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:45.412982 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.412986 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.414134 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.414135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.414891 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.414893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.415519 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.415520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.415635 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.416113 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:45.416124 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:45.419902 (+ 3778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:45.422188 (+ 2286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:45.422196 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:45.422199 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:45.422547 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:45.422832 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:45.422833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:45.422833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:45.422967 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:45.423436 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:45.423443 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:45.423589 (+ 146us) client_negotiation.cc:769] Sending connection context
0929 19:12:45.424373 (+ 784us) client_negotiation.cc:240] Negotiation successful
0929 19:12:45.424544 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
I20260929 19:12:45.425694 24618 heartbeater.cc:347] Connected to a master server at 127.9.236.62:46247
I20260929 19:12:45.425801 24618 heartbeater.cc:464] Registering TS with master...
I20260929 19:12:45.426002 24618 heartbeater.cc:511] Master 127.9.236.62:46247 requested a full tablet report, sending...
I20260929 19:12:45.426826 24430 ts_manager.cc:194] Registered new tserver with Master: b0625621a9574adca65c1fa807eb83a8 (127.9.236.1:42881)
I20260929 19:12:45.428016 24430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:51143
I20260929 19:12:45.429013 24618 heartbeater.cc:503] Master 127.9.236.62:46247 was elected leader, sending a full tablet report...
Sep 29 19:12:45 dist-test-slave-wftq krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709165, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for kudu/127.9.236.1@KRBTEST.COM
I20260929 19:12:45.437387 24626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.427752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33132 (local address 127.9.236.1:42881)
0929 19:12:45.428150 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:45.428154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:45.428163 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:12:45.428199 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:45.428201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:45.428216 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:45.428480 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:45.429078 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.430558 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.431531 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.432038 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.432116 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.434547 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:45.434554 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:45.434555 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:45.434562 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:45.436080 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:45.436515 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:45.436517 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:45.436518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:45.436566 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:45.436750 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:45.436750 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:45.436751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:45.436904 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:45.436975 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:45.437229 (+ 254us) server_negotiation.cc:299] Negotiation successful
0929 19:12:45.437320 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":78,"threads_started":1}
I20260929 19:12:45.437497 24625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.427468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:42881 (local address 127.0.0.1:33132)
0929 19:12:45.427918 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:45.427955 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:45.428029 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:45.428546 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:45.428551 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:45.428919 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:45.428995 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.428998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.430669 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.430670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.431441 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.431444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.432127 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.432128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.432213 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.432841 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:45.432846 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:45.434437 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:45.436161 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:45.436167 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:45.436169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:45.436436 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:45.436626 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:45.436627 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:45.436627 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:45.436695 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:45.436966 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:45.436971 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:45.437031 (+ 60us) client_negotiation.cc:769] Sending connection context
0929 19:12:45.437197 (+ 166us) client_negotiation.cc:240] Negotiation successful
0929 19:12:45.437338 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":116,"threads_started":1}
I20260929 19:12:45.437913 24551 tablet_service.cc:1514] Processing DeleteTablet for tablet 96bd7792985e4736b98fa9340fe365ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:12:45 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:33132
I20260929 19:12:45.438452 24627 tablet_replica.cc:330] stopping tablet replica
I20260929 19:12:45.438563 24627 raft_consensus.cc:2229] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:12:45.438642 24627 raft_consensus.cc:2258] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:12:45.439023 24627 ts_tablet_manager.cc:1905] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:12:45.441439 24627 ts_tablet_manager.cc:1918] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:12:45.441515 24627 log.cc:1196] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/wal/wals/96bd7792985e4736b98fa9340fe365ee
I20260929 19:12:45.441740 24627 ts_tablet_manager.cc:1939] T 96bd7792985e4736b98fa9340fe365ee P b0625621a9574adca65c1fa807eb83a8: Deleting consensus metadata
I20260929 19:12:45.442270 24416 catalog_manager.cc:4841] TS b0625621a9574adca65c1fa807eb83a8 (127.9.236.1:42881): tablet 96bd7792985e4736b98fa9340fe365ee (table test-table [id=9b84825f0721436989fedc409ae8a623]) successfully deleted
W20260929 19:12:45.505353 24622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:45.505590 24622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:45.505615 24622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:45.506800 24622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:45.506840 24622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:45.506865 24622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:12:45.508323 24622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:46237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:43261
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=44725
--tserver_master_addrs=127.9.236.62:46247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:45.508538 24622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:45.508747 24622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:45.513989 24632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:45.514020 24633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:45.514027 24635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:45.514102 24622 server_base.cc:1034] running on GCE node
I20260929 19:12:45.514469 24622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:45.514696 24622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:45.515872 24622 hybrid_clock.cc:648] HybridClock initialized: now 1790709165515862 us; error 31 us; skew 500 ppm
Sep 29 19:12:45 dist-test-slave-wftq krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709165, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:45.518802 24622 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:12:45.519331 24622 webserver.cc:460] Webserver started at http://127.9.236.2:44725/ using document root <none> and password file <none>
I20260929 19:12:45.519567 24622 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:45.519636 24622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:12:45.521232 24622 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:12:45.521986 24642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:45.522193 24622 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:45.522282 24622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "ca5b39449c13474f98f601cb46c0ea0b"
format_stamp: "Formatted at 2026-09-29 19:12:44 on dist-test-slave-wftq"
server_key: "4a0ef27757fda696f662487d2de1ea23"
server_key_iv: "9bbbb99f824d9879e9f6649073d03e9b"
server_key_version: "encryptionkey@0"
I20260929 19:12:45.522708 24622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:45.543603 24622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:45.545930 24622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:45.546123 24622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:45.546392 24622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:12:45.546869 24649 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 19:12:45.548375 24622 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 19:12:45.548424 24622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260929 19:12:45.548460 24622 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 19:12:45.549275 24622 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 19:12:45.549309 24622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:45.549403 24649 tablet_bootstrap.cc:492] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b: Bootstrap starting.
I20260929 19:12:45.557078 24622 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:43261
I20260929 19:12:45.557227 24756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:43261 every 8 connection(s)
I20260929 19:12:45.557556 24622 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:12:45.558043 24649 log.cc:824] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b: Log is configured to *not* fsync() on all Append() calls
I20260929 19:12:45.560380 24649 tablet_bootstrap.cc:492] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:12:45.560654 24649 tablet_bootstrap.cc:492] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b: Bootstrap complete.
I20260929 19:12:45.561780 24649 ts_tablet_manager.cc:1397] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.011s
I20260929 19:12:45.563171 24649 raft_consensus.cc:348] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } } peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } }
I20260929 19:12:45.563503 24649 raft_consensus.cc:729] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5b39449c13474f98f601cb46c0ea0b, State: Initialized, Role: FOLLOWER
I20260929 19:12:45.563649 24649 consensus_queue.cc:260] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } } peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } }
I20260929 19:12:45.563932 24649 ts_tablet_manager.cc:1428] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
Sep 29 19:12:45 dist-test-slave-wftq krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709165, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:12:45.567099 24621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.558402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:60571 (local address 127.9.236.62:46247)
0929 19:12:45.558481 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:45.558484 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:45.558677 (+ 193us) server_negotiation.cc:407] Connection header received
0929 19:12:45.559205 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:45.559207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:45.559224 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:45.559288 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:45.559928 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.560432 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.562027 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.562223 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.562323 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.564801 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:45.564802 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:45.564802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:45.564808 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:45.566025 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:45.566377 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:45.566378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:45.566378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:45.566421 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:45.566610 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:45.566611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:45.566611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:45.566702 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:45.566751 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:45.566974 (+ 223us) server_negotiation.cc:299] Negotiation successful
0929 19:12:45.567034 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 19:12:45.567291 24760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.558332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.2:60571)
0929 19:12:45.558518 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:45.558566 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:45.559107 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:45.559354 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:45.559359 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:45.559631 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:45.559842 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.559849 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.560558 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.560560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.561923 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.561927 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.562359 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.562360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.562528 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.563084 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:45.563094 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:45.564712 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:45.566096 (+ 1384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:45.566102 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:45.566104 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:45.566315 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:45.566472 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:45.566472 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:45.566472 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:45.566562 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:45.566751 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:45.566754 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:45.566870 (+ 116us) client_negotiation.cc:769] Sending connection context
0929 19:12:45.566951 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 19:12:45.567087 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260929 19:12:45.567550 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 24622
I20260929 19:12:45.567816 24757 heartbeater.cc:347] Connected to a master server at 127.9.236.62:46247
I20260929 19:12:45.567859 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:40443
--local_ip_for_outbound_sockets=127.9.236.3
--tserver_master_addrs=127.9.236.62:46247
--webserver_port=42143
--webserver_interface=127.9.236.3
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:46237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:12:45.567911 24757 heartbeater.cc:464] Registering TS with master...
I20260929 19:12:45.568104 24757 heartbeater.cc:511] Master 127.9.236.62:46247 requested a full tablet report, sending...
I20260929 19:12:45.568719 24430 ts_manager.cc:194] Registered new tserver with Master: ca5b39449c13474f98f601cb46c0ea0b (127.9.236.2:43261)
I20260929 19:12:45.569300 24430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:60571
I20260929 19:12:45.570117 24757 heartbeater.cc:503] Master 127.9.236.62:46247 was elected leader, sending a full tablet report...
Sep 29 19:12:45 dist-test-slave-wftq krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709165, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for kudu/127.9.236.2@KRBTEST.COM
I20260929 19:12:45.577188 24625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.569030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:43261 (local address 127.0.0.1:54098)
0929 19:12:45.569249 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:45.569261 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:45.569314 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:45.569470 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:45.569471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:45.569642 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:45.569701 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.569704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.571185 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.571186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.571729 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.571731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.572366 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.572366 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.572412 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.572892 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:45.572898 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:45.574274 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:45.576116 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:45.576117 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:45.576117 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:45.576287 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:45.576504 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:45.576504 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:45.576505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:45.576557 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:45.576875 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:45.576875 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:45.576911 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 19:12:45.576958 (+ 47us) client_negotiation.cc:240] Negotiation successful
0929 19:12:45.577061 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199}
I20260929 19:12:45.577245 24763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.569080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54098 (local address 127.9.236.2:43261)
0929 19:12:45.569262 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:45.569267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:45.569300 (+ 33us) server_negotiation.cc:407] Connection header received
0929 19:12:45.569381 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:45.569383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:45.569398 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:45.569458 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:45.569795 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.571086 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.571812 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.572288 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.572367 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.574361 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:45.574369 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:45.574369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:45.574376 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:45.576031 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:45.576369 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:45.576372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:45.576373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:45.576436 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:45.576615 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:45.576616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:45.576617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:45.576803 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:45.576906 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:45.577005 (+ 99us) server_negotiation.cc:299] Negotiation successful
0929 19:12:45.577135 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":152,"thread_start_us":111,"threads_started":1}
I20260929 19:12:45.577667 24691 tablet_service.cc:1514] Processing DeleteTablet for tablet 96bd7792985e4736b98fa9340fe365ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:12:45 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:54098
I20260929 19:12:45.578248 24764 tablet_replica.cc:330] stopping tablet replica
I20260929 19:12:45.578349 24764 raft_consensus.cc:2229] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:12:45.578426 24764 raft_consensus.cc:2258] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:12:45.578832 24764 ts_tablet_manager.cc:1905] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:12:45.581091 24764 ts_tablet_manager.cc:1918] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:12:45.581220 24764 log.cc:1196] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/96bd7792985e4736b98fa9340fe365ee
I20260929 19:12:45.581453 24764 ts_tablet_manager.cc:1939] T 96bd7792985e4736b98fa9340fe365ee P ca5b39449c13474f98f601cb46c0ea0b: Deleting consensus metadata
I20260929 19:12:45.582070 24414 catalog_manager.cc:4841] TS ca5b39449c13474f98f601cb46c0ea0b (127.9.236.2:43261): tablet 96bd7792985e4736b98fa9340fe365ee (table test-table [id=9b84825f0721436989fedc409ae8a623]) successfully deleted
W20260929 19:12:45.652801 24762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:45.652994 24762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:45.653023 24762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:45.654162 24762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:45.654204 24762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:45.654237 24762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:12:45.655632 24762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:46237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:40443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=42143
--tserver_master_addrs=127.9.236.62:46247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:45.655856 24762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:45.656059 24762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:45.661536 24770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:45.661623 24769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:45.661624 24772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:45.661908 24762 server_base.cc:1034] running on GCE node
I20260929 19:12:45.662120 24762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:45.662362 24762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:45.663511 24762 hybrid_clock.cc:648] HybridClock initialized: now 1790709165663508 us; error 65 us; skew 500 ppm
Sep 29 19:12:45 dist-test-slave-wftq krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709165, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:45.666280 24762 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:12:45.666837 24762 webserver.cc:460] Webserver started at http://127.9.236.3:42143/ using document root <none> and password file <none>
I20260929 19:12:45.667076 24762 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:45.667143 24762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:12:45.668727 24762 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:12:45.669458 24779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:45.669715 24762 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:45.669818 24762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "90886c62688a4d4c92ebf9f840f0b5d2"
format_stamp: "Formatted at 2026-09-29 19:12:44 on dist-test-slave-wftq"
server_key: "1d890a1c5f8135852767dbfcfb5238dc"
server_key_iv: "2f25a1e23de69651bdc13cc0b0ffcf95"
server_key_version: "encryptionkey@0"
I20260929 19:12:45.670251 24762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:45.684517 24762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:45.686867 24762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:45.687016 24762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:45.687269 24762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:12:45.687611 24762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:12:45.687645 24762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:45.687670 24762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:12:45.687690 24762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:45.694456 24762 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:40443
I20260929 19:12:45.694554 24892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:40443 every 8 connection(s)
I20260929 19:12:45.694871 24762 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:12:45.703292 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 24762
Sep 29 19:12:45 dist-test-slave-wftq krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709165, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:12:45.709273 24621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.695619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:57293 (local address 127.9.236.62:46247)
0929 19:12:45.695713 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:45.695716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:45.695967 (+ 251us) server_negotiation.cc:407] Connection header received
0929 19:12:45.696623 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:45.696625 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:45.696636 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:45.696697 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:45.697795 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.698285 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.699430 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.699617 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.699672 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.706713 (+ 7041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:45.706716 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:45.706716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:45.706725 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:45.707999 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:45.708367 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:45.708368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:45.708369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:45.708416 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:45.708620 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:45.708621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:45.708621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:45.708732 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:45.708788 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:45.709050 (+ 262us) server_negotiation.cc:299] Negotiation successful
0929 19:12:45.709172 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 19:12:45.709431 24895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.695591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.3:57293)
0929 19:12:45.695799 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:45.695840 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:45.696511 (+ 671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:45.696778 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:45.696781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:45.697417 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:45.697714 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.697719 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.698367 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.698368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.699352 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.699354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.699718 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.699719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.699855 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.705018 (+ 5163us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:45.705029 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:45.706614 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:45.708076 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:45.708083 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:45.708086 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:45.708302 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:45.708468 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:45.708469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:45.708470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:45.708567 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:45.708788 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:45.708794 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:45.708937 (+ 143us) client_negotiation.cc:769] Sending connection context
0929 19:12:45.709022 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 19:12:45.709211 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":119,"threads_started":1}
I20260929 19:12:45.710000 24893 heartbeater.cc:347] Connected to a master server at 127.9.236.62:46247
I20260929 19:12:45.710098 24893 heartbeater.cc:464] Registering TS with master...
I20260929 19:12:45.710292 24893 heartbeater.cc:511] Master 127.9.236.62:46247 requested a full tablet report, sending...
I20260929 19:12:45.710767 24430 ts_manager.cc:194] Registered new tserver with Master: 90886c62688a4d4c92ebf9f840f0b5d2 (127.9.236.3:40443)
I20260929 19:12:45.711223 24430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:57293
I20260929 19:12:45.711323 24897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.703784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40998 (local address 127.9.236.62:46247)
0929 19:12:45.703972 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:45.703973 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:45.703984 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:12:45.704059 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:45.704060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:45.704073 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:45.704130 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:45.704721 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.705364 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.706175 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.706424 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.706485 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.707890 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:45.707891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:45.707891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:45.707898 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:45.709344 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:45.710488 (+ 1144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:45.710489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:45.710489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:45.710540 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:45.710753 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:45.710754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:45.710754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:45.710888 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:45.711024 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:45.711115 (+ 91us) server_negotiation.cc:299] Negotiation successful
0929 19:12:45.711184 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":99,"threads_started":1}
I20260929 19:12:45.712091 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:12:45.714920 24897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.712452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41014 (local address 127.9.236.62:46247)
0929 19:12:45.712514 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:45.712516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:45.712625 (+ 109us) server_negotiation.cc:407] Connection header received
0929 19:12:45.712699 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:45.712700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:45.712730 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:45.712791 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:12:45.712919 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.713480 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.714213 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.714425 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.714480 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.714642 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:12:45.714742 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:12:45.714812 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:45.714834 (+ 22us) server_negotiation.cc:299] Negotiation successful
0929 19:12:45.714857 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 19:12:45.715231 24430 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:41014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:12:45.717140 24430 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:12:45.719389 24691 tablet_service.cc:1467] Processing CreateTablet for tablet d5045d013792492d93ccf836c07a4622 (DEFAULT_TABLE table=test-table [id=386a1c0b01f3455a85c19b1a5c15076d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:12:45.719384 24551 tablet_service.cc:1467] Processing CreateTablet for tablet d5045d013792492d93ccf836c07a4622 (DEFAULT_TABLE table=test-table [id=386a1c0b01f3455a85c19b1a5c15076d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:12:45.719604 24691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5045d013792492d93ccf836c07a4622. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:12:45.719604 24551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5045d013792492d93ccf836c07a4622. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:12:45.721381 24510 tablet_bootstrap.cc:492] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8: Bootstrap starting.
I20260929 19:12:45.721381 24649 tablet_bootstrap.cc:492] T d5045d013792492d93ccf836c07a4622 P ca5b39449c13474f98f601cb46c0ea0b: Bootstrap starting.
I20260929 19:12:45.721849 24510 tablet_bootstrap.cc:654] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8: Neither blocks nor log segments found. Creating new log.
I20260929 19:12:45.721849 24649 tablet_bootstrap.cc:654] T d5045d013792492d93ccf836c07a4622 P ca5b39449c13474f98f601cb46c0ea0b: Neither blocks nor log segments found. Creating new log.
I20260929 19:12:45.722708 24649 tablet_bootstrap.cc:492] T d5045d013792492d93ccf836c07a4622 P ca5b39449c13474f98f601cb46c0ea0b: No bootstrap required, opened a new log
I20260929 19:12:45.722697 24625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.719083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:40443 (local address 127.0.0.1:51466)
0929 19:12:45.719168 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:45.719178 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:45.719211 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:45.719677 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:45.719679 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:45.719680 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:12:45.719731 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.719733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.720651 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.720652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.721685 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.721688 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.722467 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.722468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.722531 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.722549 (+ 18us) client_negotiation.cc:769] Sending connection context
0929 19:12:45.722580 (+ 31us) client_negotiation.cc:240] Negotiation successful
0929 19:12:45.722623 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260929 19:12:45.722755 24649 ts_tablet_manager.cc:1397] T d5045d013792492d93ccf836c07a4622 P ca5b39449c13474f98f601cb46c0ea0b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260929 19:12:45.722980 24510 tablet_bootstrap.cc:492] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8: No bootstrap required, opened a new log
I20260929 19:12:45.723050 24510 ts_tablet_manager.cc:1397] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:12:45.723083 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.719289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51466 (local address 127.9.236.3:40443)
0929 19:12:45.719466 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:45.719471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:45.719481 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:12:45.719520 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:45.719522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:45.719537 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:45.719813 (+ 276us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:12:45.719866 (+ 53us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.720562 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.721757 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.722368 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.722875 (+ 507us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.722947 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:45.722972 (+ 25us) server_negotiation.cc:299] Negotiation successful
0929 19:12:45.723013 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":96,"threads_started":1}
I20260929 19:12:45.723114 24649 raft_consensus.cc:348] T d5045d013792492d93ccf836c07a4622 P ca5b39449c13474f98f601cb46c0ea0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } } peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } }
I20260929 19:12:45.723201 24649 raft_consensus.cc:374] T d5045d013792492d93ccf836c07a4622 P ca5b39449c13474f98f601cb46c0ea0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:12:45.723241 24649 raft_consensus.cc:729] T d5045d013792492d93ccf836c07a4622 P ca5b39449c13474f98f601cb46c0ea0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5b39449c13474f98f601cb46c0ea0b, State: Initialized, Role: FOLLOWER
I20260929 19:12:45.723289 24649 consensus_queue.cc:260] T d5045d013792492d93ccf836c07a4622 P ca5b39449c13474f98f601cb46c0ea0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } } peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } }
I20260929 19:12:45.723251 24510 raft_consensus.cc:348] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } } peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } }
I20260929 19:12:45.723379 24649 ts_tablet_manager.cc:1428] T d5045d013792492d93ccf836c07a4622 P ca5b39449c13474f98f601cb46c0ea0b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260929 19:12:45.723341 24510 raft_consensus.cc:374] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:12:45.723413 24510 raft_consensus.cc:729] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0625621a9574adca65c1fa807eb83a8, State: Initialized, Role: FOLLOWER
I20260929 19:12:45.723474 24510 consensus_queue.cc:260] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } } peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } }
I20260929 19:12:45.723565 24510 ts_tablet_manager.cc:1428] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:45.723865 24827 tablet_service.cc:1467] Processing CreateTablet for tablet d5045d013792492d93ccf836c07a4622 (DEFAULT_TABLE table=test-table [id=386a1c0b01f3455a85c19b1a5c15076d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:12:45.724229 24827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5045d013792492d93ccf836c07a4622. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:12:45.727432 24902 tablet_bootstrap.cc:492] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2: Bootstrap starting.
I20260929 19:12:45.728041 24902 tablet_bootstrap.cc:654] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2: Neither blocks nor log segments found. Creating new log.
I20260929 19:12:45.728483 24902 log.cc:824] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2: Log is configured to *not* fsync() on all Append() calls
I20260929 19:12:45.729369 24902 tablet_bootstrap.cc:492] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2: No bootstrap required, opened a new log
I20260929 19:12:45.729432 24902 ts_tablet_manager.cc:1397] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:12:45.730832 24902 raft_consensus.cc:348] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } } peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } }
I20260929 19:12:45.730967 24902 raft_consensus.cc:374] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:12:45.730988 24902 raft_consensus.cc:729] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90886c62688a4d4c92ebf9f840f0b5d2, State: Initialized, Role: FOLLOWER
I20260929 19:12:45.731070 24902 consensus_queue.cc:260] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } } peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } }
I20260929 19:12:45.731279 24902 ts_tablet_manager.cc:1428] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:12:45.731364 24893 heartbeater.cc:503] Master 127.9.236.62:46247 was elected leader, sending a full tablet report...
I20260929 19:12:45.763553 24904 raft_consensus.cc:482] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:12:45.763749 24904 raft_consensus.cc:504] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } } peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } }
I20260929 19:12:45.764248 24904 leader_election.cc:288] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca5b39449c13474f98f601cb46c0ea0b (127.9.236.2:43261), b0625621a9574adca65c1fa807eb83a8 (127.9.236.1:42881)
I20260929 19:12:45.767759 24895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.764447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:43261 (local address 127.9.236.3:58773)
0929 19:12:45.764518 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:45.764534 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:45.764617 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:45.764847 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:45.764849 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:45.764851 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:12:45.764931 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.764932 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.765710 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.765711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.766577 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.766579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.767422 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.767423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.767546 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.767551 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:12:45.767611 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 19:12:45.767668 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 19:12:45.767823 24763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.764573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:58773 (local address 127.9.236.2:43261)
0929 19:12:45.764649 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:45.764651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:45.764662 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:12:45.764696 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:45.764698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:45.764730 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:45.764814 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:12:45.765017 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.765627 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.766658 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.767334 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.767412 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.767483 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:45.767687 (+ 204us) server_negotiation.cc:299] Negotiation successful
0929 19:12:45.767754 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 19:12:45.767874 24905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.764447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:42881 (local address 127.9.236.3:60277)
0929 19:12:45.764746 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:45.764755 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:45.764820 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:45.765003 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:45.765004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:45.765007 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:12:45.765093 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.765095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.765793 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.765794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.766647 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:45.766650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.767545 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.767546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:45.767631 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.767637 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 19:12:45.767712 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 19:12:45.767771 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":83,"thread_start_us":125,"threads_started":1}
I20260929 19:12:45.767854 24626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.764560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:60277 (local address 127.9.236.1:42881)
0929 19:12:45.764674 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:45.764676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:45.764830 (+ 154us) server_negotiation.cc:407] Connection header received
0929 19:12:45.764882 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:45.764883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:45.764914 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:45.764989 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:12:45.765178 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.765696 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.766732 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.767336 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.767412 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.767484 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:45.767725 (+ 241us) server_negotiation.cc:299] Negotiation successful
0929 19:12:45.767777 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 19:12:45.768224 24572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5045d013792492d93ccf836c07a4622" candidate_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0625621a9574adca65c1fa807eb83a8" is_pre_election: true
I20260929 19:12:45.768241 24711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5045d013792492d93ccf836c07a4622" candidate_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5b39449c13474f98f601cb46c0ea0b" is_pre_election: true
I20260929 19:12:45.768390 24711 raft_consensus.cc:2454] T d5045d013792492d93ccf836c07a4622 P ca5b39449c13474f98f601cb46c0ea0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90886c62688a4d4c92ebf9f840f0b5d2 in term 0.
I20260929 19:12:45.768409 24572 raft_consensus.cc:2454] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90886c62688a4d4c92ebf9f840f0b5d2 in term 0.
I20260929 19:12:45.768656 24782 leader_election.cc:302] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90886c62688a4d4c92ebf9f840f0b5d2, b0625621a9574adca65c1fa807eb83a8; no voters:
I20260929 19:12:45.768826 24904 raft_consensus.cc:2781] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:12:45.768887 24904 raft_consensus.cc:482] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:12:45.768906 24904 raft_consensus.cc:3037] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:12:45.769665 24904 raft_consensus.cc:504] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } } peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } }
I20260929 19:12:45.769809 24904 leader_election.cc:288] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 [CANDIDATE]: Term 1 election: Requested vote from peers ca5b39449c13474f98f601cb46c0ea0b (127.9.236.2:43261), b0625621a9574adca65c1fa807eb83a8 (127.9.236.1:42881)
I20260929 19:12:45.769984 24711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5045d013792492d93ccf836c07a4622" candidate_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5b39449c13474f98f601cb46c0ea0b"
I20260929 19:12:45.770018 24572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5045d013792492d93ccf836c07a4622" candidate_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0625621a9574adca65c1fa807eb83a8"
I20260929 19:12:45.770087 24572 raft_consensus.cc:3037] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:12:45.770087 24711 raft_consensus.cc:3037] T d5045d013792492d93ccf836c07a4622 P ca5b39449c13474f98f601cb46c0ea0b [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:12:45.771099 24572 raft_consensus.cc:2454] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90886c62688a4d4c92ebf9f840f0b5d2 in term 1.
I20260929 19:12:45.771099 24711 raft_consensus.cc:2454] T d5045d013792492d93ccf836c07a4622 P ca5b39449c13474f98f601cb46c0ea0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90886c62688a4d4c92ebf9f840f0b5d2 in term 1.
I20260929 19:12:45.771284 24782 leader_election.cc:302] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 90886c62688a4d4c92ebf9f840f0b5d2, b0625621a9574adca65c1fa807eb83a8, ca5b39449c13474f98f601cb46c0ea0b; no voters:
I20260929 19:12:45.771391 24904 raft_consensus.cc:2781] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:12:45.771550 24904 raft_consensus.cc:686] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 [term 1 LEADER]: Becoming Leader. State: Replica: 90886c62688a4d4c92ebf9f840f0b5d2, State: Running, Role: LEADER
I20260929 19:12:45.771695 24904 consensus_queue.cc:237] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } } peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } }
I20260929 19:12:45.772581 24428 catalog_manager.cc:5495] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90886c62688a4d4c92ebf9f840f0b5d2 (127.9.236.3). New cstate: current_term: 1 leader_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } health_report { overall_health: UNKNOWN } } }
I20260929 19:12:45.784013 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.781798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51470 (local address 127.9.236.3:40443)
0929 19:12:45.781851 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:45.781853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:45.781889 (+ 36us) server_negotiation.cc:407] Connection header received
0929 19:12:45.781915 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:45.781915 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:45.781934 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:45.781991 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:12:45.782128 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.782587 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.783365 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:45.783543 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:45.783593 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:45.783705 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:12:45.783805 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:12:45.783866 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:45.783910 (+ 44us) server_negotiation.cc:299] Negotiation successful
0929 19:12:45.783927 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260929 19:12:45.786162 24711 raft_consensus.cc:1261] T d5045d013792492d93ccf836c07a4622 P ca5b39449c13474f98f601cb46c0ea0b [term 1 FOLLOWER]: Refusing update from remote peer 90886c62688a4d4c92ebf9f840f0b5d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:12:45.786191 24572 raft_consensus.cc:1261] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Refusing update from remote peer 90886c62688a4d4c92ebf9f840f0b5d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:12:45.786562 24906 consensus_queue.cc:1035] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:12:45.786724 24904 consensus_queue.cc:1035] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:12:45.789549 24910 mvcc.cc:205] Tried to move back new op lower bound from 7334744743057059840 to 7334744743001640960. Current Snapshot: MvccSnapshot[applied={T|T < 7334744743057059840}]
I20260929 19:12:45.791800 24430 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:41014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:12:45.791898 24430 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:41014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:12:45.792680 24430 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe: Sending DeleteTablet for 3 replicas of tablet d5045d013792492d93ccf836c07a4622
I20260929 19:12:45.792963 24551 tablet_service.cc:1514] Processing DeleteTablet for tablet d5045d013792492d93ccf836c07a4622 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:12:45 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:33132
I20260929 19:12:45.792994 24827 tablet_service.cc:1514] Processing DeleteTablet for tablet d5045d013792492d93ccf836c07a4622 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:12:45 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:51466
I20260929 19:12:45.793069 24627 tablet_replica.cc:330] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8: stopping tablet replica
I20260929 19:12:45.793035 24691 tablet_service.cc:1514] Processing DeleteTablet for tablet d5045d013792492d93ccf836c07a4622 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:12:45 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:54098
I20260929 19:12:45.793146 24627 raft_consensus.cc:2229] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:12:45.793174 24764 tablet_replica.cc:330] T d5045d013792492d93ccf836c07a4622 P ca5b39449c13474f98f601cb46c0ea0b: stopping tablet replica
I20260929 19:12:45.793196 24627 raft_consensus.cc:2258] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:12:45.793227 24764 raft_consensus.cc:2229] T d5045d013792492d93ccf836c07a4622 P ca5b39449c13474f98f601cb46c0ea0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:12:45.793310 24764 raft_consensus.cc:2258] T d5045d013792492d93ccf836c07a4622 P ca5b39449c13474f98f601cb46c0ea0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:12:45.793411 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 24468
I20260929 19:12:45.793546 24627 ts_tablet_manager.cc:1905] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:12:45.797317 24918 tablet_replica.cc:330] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2: stopping tablet replica
I20260929 19:12:45.797451 24764 ts_tablet_manager.cc:1905] T d5045d013792492d93ccf836c07a4622 P ca5b39449c13474f98f601cb46c0ea0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:12:45.797632 24918 raft_consensus.cc:2229] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:12:45.797992 24918 raft_consensus.cc:2258] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:12:45.798631 24918 ts_tablet_manager.cc:1905] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 19:12:45.798893 24416 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.236.1:42881 (error 108)
I20260929 19:12:45.799108 24764 ts_tablet_manager.cc:1918] T d5045d013792492d93ccf836c07a4622 P ca5b39449c13474f98f601cb46c0ea0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:12:45.799157 24764 log.cc:1196] T d5045d013792492d93ccf836c07a4622 P ca5b39449c13474f98f601cb46c0ea0b: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/d5045d013792492d93ccf836c07a4622
I20260929 19:12:45.799373 24764 ts_tablet_manager.cc:1939] T d5045d013792492d93ccf836c07a4622 P ca5b39449c13474f98f601cb46c0ea0b: Deleting consensus metadata
I20260929 19:12:45.799625 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 24622
I20260929 19:12:45.799654 24414 catalog_manager.cc:4841] TS ca5b39449c13474f98f601cb46c0ea0b (127.9.236.2:43261): tablet d5045d013792492d93ccf836c07a4622 (table test-table [id=386a1c0b01f3455a85c19b1a5c15076d]) successfully deleted
I20260929 19:12:45.800876 24918 ts_tablet_manager.cc:1918] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:12:45.800962 24918 log.cc:1196] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/d5045d013792492d93ccf836c07a4622
I20260929 19:12:45.801329 24918 ts_tablet_manager.cc:1939] T d5045d013792492d93ccf836c07a4622 P 90886c62688a4d4c92ebf9f840f0b5d2: Deleting consensus metadata
I20260929 19:12:45.801790 24414 catalog_manager.cc:4841] TS 90886c62688a4d4c92ebf9f840f0b5d2 (127.9.236.3:40443): tablet d5045d013792492d93ccf836c07a4622 (table test-table [id=386a1c0b01f3455a85c19b1a5c15076d]) successfully deleted
I20260929 19:12:45.804663 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 24762
I20260929 19:12:45.805433 24625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:45.804876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:42881 (local address 127.0.0.1:33144)
0929 19:12:45.805358 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:45.805403 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.1:42881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459}
W20260929 19:12:45.806150 24416 catalog_manager.cc:4584] TS b0625621a9574adca65c1fa807eb83a8 (127.9.236.1:42881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d5045d013792492d93ccf836c07a4622: Network error: Client connection negotiation failed: client connection to 127.9.236.1:42881: connect: Connection refused (error 111)
I20260929 19:12:45.810374 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 24397
I20260929 19:12:45.815840 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:46247
--webserver_interface=127.9.236.62
--webserver_port=36971
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:46237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:46247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:12:45.894937 24920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:45.895144 24920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:45.895172 24920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:45.896327 24920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:12:45.896365 24920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:45.896382 24920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:45.896397 24920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:12:45.896412 24920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:12:45.897873 24920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:46237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:46247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:46247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=36971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:45.898109 24920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:45.898313 24920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:45.903178 24926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:45.903288 24925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:45.903172 24928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:45.903301 24920 server_base.cc:1034] running on GCE node
I20260929 19:12:45.903615 24920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:45.903857 24920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:45.904981 24920 hybrid_clock.cc:648] HybridClock initialized: now 1790709165904958 us; error 45 us; skew 500 ppm
Sep 29 19:12:45 dist-test-slave-wftq krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709165, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:45.907393 24920 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:12:45.907832 24920 webserver.cc:460] Webserver started at http://127.9.236.62:36971/ using document root <none> and password file <none>
I20260929 19:12:45.908043 24920 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:45.908092 24920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:12:45.909437 24920 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:12:45.910135 24935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:45.910337 24920 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:12:45.910421 24920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "5e59deb6c22c4d20ba8546127ccb13fe"
format_stamp: "Formatted at 2026-09-29 19:12:44 on dist-test-slave-wftq"
server_key: "57b1faa641d7082f1e04a0d1af39986b"
server_key_iv: "ac3ad01b4db3de1e38e80148686e4d18"
server_key_version: "encryptionkey@0"
I20260929 19:12:45.910745 24920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:45.921825 24920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:45.923681 24920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:45.923812 24920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:45.927795 24920 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:46247
I20260929 19:12:45.927862 24987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:46247 every 8 connection(s)
I20260929 19:12:45.928181 24920 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:12:45.930105 24988 sys_catalog.cc:263] Verifying existing consensus state
I20260929 19:12:45.930835 24988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe: Bootstrap starting.
I20260929 19:12:45.931509 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 24920
I20260929 19:12:45.931807 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:42881
--local_ip_for_outbound_sockets=127.9.236.1
--tserver_master_addrs=127.9.236.62:46247
--webserver_port=41895
--webserver_interface=127.9.236.1
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:46237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:12:45.941638 24988 log.cc:824] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe: Log is configured to *not* fsync() on all Append() calls
I20260929 19:12:45.944082 24988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:12:45.944377 24988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe: Bootstrap complete.
I20260929 19:12:45.946802 24988 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46247 } }
I20260929 19:12:45.947054 24988 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e59deb6c22c4d20ba8546127ccb13fe, State: Initialized, Role: FOLLOWER
I20260929 19:12:45.947219 24988 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46247 } }
I20260929 19:12:45.947293 24988 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:12:45.947342 24988 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:12:45.947388 24988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 2 FOLLOWER]: Advancing to term 3
I20260929 19:12:45.948493 24988 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46247 } }
I20260929 19:12:45.948594 24988 leader_election.cc:302] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e59deb6c22c4d20ba8546127ccb13fe; no voters:
I20260929 19:12:45.948781 24988 leader_election.cc:288] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 19:12:45.948894 24992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 3 FOLLOWER]: Leader election won for term 3
I20260929 19:12:45.949074 24988 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:12:45.949167 24992 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [term 3 LEADER]: Becoming Leader. State: Replica: 5e59deb6c22c4d20ba8546127ccb13fe, State: Running, Role: LEADER
I20260929 19:12:45.949317 24992 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46247 } }
I20260929 19:12:45.949700 24992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e59deb6c22c4d20ba8546127ccb13fe. Latest consensus state: current_term: 3 leader_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46247 } } }
I20260929 19:12:45.949782 24992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [sys.catalog]: This master's current role is: LEADER
I20260929 19:12:45.949736 24993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e59deb6c22c4d20ba8546127ccb13fe" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46247 } } }
I20260929 19:12:45.949842 24993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e59deb6c22c4d20ba8546127ccb13fe [sys.catalog]: This master's current role is: LEADER
I20260929 19:12:45.950527 25001 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:12:45.951624 25001 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:12:45.951728 25001 catalog_manager.cc:1259] Loaded cluster ID: 545d72676b5444cd87c4aae388c586d3
I20260929 19:12:45.951771 25001 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260929 19:12:45.951990 25001 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1789ab99ad google::LogMessage::Fail() at ??:0
@ 0x7f1789abc88a google::LogMessage::SendToLog() at ??:0
@ 0x7f1789ab94de google::LogMessage::Flush() at ??:0
@ 0x7f1789abaa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f17919da0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f17919e38b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f178b9870d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f178b97c954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f178c7e06db start_thread at ??:0
@ 0x7f17885b761f clone at ??:0
W20260929 19:12:46.025761 24990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:46.025970 24990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:46.025993 24990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:46.027181 24990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:46.027230 24990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:46.027266 24990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:12:46.028718 24990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:46237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:42881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=41895
--tserver_master_addrs=127.9.236.62:46247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:46.028940 24990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:46.029192 24990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:12:46.035614 25025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:46.035687 25024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:46.035890 24990 server_base.cc:1034] running on GCE node
W20260929 19:12:46.035638 25027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:46.036176 24990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:46.036405 24990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:46.037534 24990 hybrid_clock.cc:648] HybridClock initialized: now 1790709166037517 us; error 40 us; skew 500 ppm
Sep 29 19:12:46 dist-test-slave-wftq krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709166, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:46.040097 24990 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:12:46.040534 24990 webserver.cc:460] Webserver started at http://127.9.236.1:41895/ using document root <none> and password file <none>
I20260929 19:12:46.040746 24990 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:46.040795 24990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:12:46.042047 24990 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:46.042694 25034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:46.042829 24990 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:12:46.042914 24990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "b0625621a9574adca65c1fa807eb83a8"
format_stamp: "Formatted at 2026-09-29 19:12:44 on dist-test-slave-wftq"
server_key: "52346534a14ae74473aa14585c72af58"
server_key_iv: "75e1a27b9083bf3701d9bc37530b68f6"
server_key_version: "encryptionkey@0"
I20260929 19:12:46.043246 24990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:46.071440 24990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:46.073707 24990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:46.073819 24990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:46.074054 24990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:12:46.074534 25041 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 19:12:46.075904 24990 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 19:12:46.075953 24990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260929 19:12:46.075975 24990 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 19:12:46.076573 24990 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 19:12:46.076618 24990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260929 19:12:46.076673 25041 tablet_bootstrap.cc:492] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8: Bootstrap starting.
I20260929 19:12:46.078542 25041 log.cc:824] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8: Log is configured to *not* fsync() on all Append() calls
I20260929 19:12:46.082099 25041 tablet_bootstrap.cc:492] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:12:46.082535 25041 tablet_bootstrap.cc:492] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8: Bootstrap complete.
I20260929 19:12:46.082752 25041 ts_tablet_manager.cc:1397] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260929 19:12:46.083513 24990 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:42881
I20260929 19:12:46.083613 25149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:42881 every 8 connection(s)
I20260929 19:12:46.083875 24990 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:12:46.084856 25041 raft_consensus.cc:348] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } } peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } }
I20260929 19:12:46.085253 25041 raft_consensus.cc:729] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0625621a9574adca65c1fa807eb83a8, State: Initialized, Role: FOLLOWER
I20260929 19:12:46.085425 25041 consensus_queue.cc:260] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } } peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } }
I20260929 19:12:46.085692 25041 ts_tablet_manager.cc:1428] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:12:46.088195 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 24990
I20260929 19:12:46.088456 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:43261
--local_ip_for_outbound_sockets=127.9.236.2
--tserver_master_addrs=127.9.236.62:46247
--webserver_port=44725
--webserver_interface=127.9.236.2
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:46237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:12:46.144951 25152 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:12:46.084580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.1:37769)
0929 19:12:46.084765 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:46.084818 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:46.085084 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:46.144701 (+ 59617us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":72,"threads_started":1}
W20260929 19:12:46.145311 25150 heartbeater.cc:644] Failed to heartbeat to 127.9.236.62:46247 (0 consecutive failures): Network error: Failed to ping master at 127.9.236.62:46247: Client connection negotiation failed: client connection to 127.9.236.62:46247: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 19:12:46.145767 25152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:46.145596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.1:35715)
0929 19:12:46.145690 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:46.145738 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260929 19:12:46.146168 25152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:46.146076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.1:50433)
0929 19:12:46.146123 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:46.146149 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260929 19:12:46.146313 25150 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 19:12:46.176584 25154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:46.176792 25154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:46.176815 25154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:46.178028 25154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:46.178071 25154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:46.178098 25154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:12:46.179580 25154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:46237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:43261
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=44725
--tserver_master_addrs=127.9.236.62:46247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:46.179804 25154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:46.180023 25154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:12:46.185436 25154 server_base.cc:1034] running on GCE node
W20260929 19:12:46.185398 25162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:46.185397 25160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:46.185397 25159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:46.185949 25154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:46.186182 25154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:46.187314 25154 hybrid_clock.cc:648] HybridClock initialized: now 1790709166187305 us; error 29 us; skew 500 ppm
Sep 29 19:12:46 dist-test-slave-wftq krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709166, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:46.189965 25154 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:12:46.190443 25154 webserver.cc:460] Webserver started at http://127.9.236.2:44725/ using document root <none> and password file <none>
I20260929 19:12:46.190682 25154 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:46.190734 25154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:12:46.192144 25154 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:46.192770 25169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:46.192968 25154 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:46.193045 25154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "ca5b39449c13474f98f601cb46c0ea0b"
format_stamp: "Formatted at 2026-09-29 19:12:44 on dist-test-slave-wftq"
server_key: "4a0ef27757fda696f662487d2de1ea23"
server_key_iv: "9bbbb99f824d9879e9f6649073d03e9b"
server_key_version: "encryptionkey@0"
I20260929 19:12:46.193485 25154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:46.202647 25154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:46.204661 25154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:46.204797 25154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:46.205013 25154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:12:46.205360 25154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:12:46.205394 25154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:46.205425 25154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:12:46.205451 25154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:46.212364 25154 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:43261
I20260929 19:12:46.212435 25282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:43261 every 8 connection(s)
I20260929 19:12:46.212772 25154 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:12:46.213892 25285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:46.213478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.2:52193)
0929 19:12:46.213656 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:46.213742 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":95,"threads_started":1}
I20260929 19:12:46.214205 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 25154
W20260929 19:12:46.214286 25283 heartbeater.cc:644] Failed to heartbeat to 127.9.236.62:46247 (0 consecutive failures): Network error: Failed to ping master at 127.9.236.62:46247: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
I20260929 19:12:46.214480 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:40443
--local_ip_for_outbound_sockets=127.9.236.3
--tserver_master_addrs=127.9.236.62:46247
--webserver_port=42143
--webserver_interface=127.9.236.3
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:46237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:12:46.214607 25285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:46.214491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.2:38685)
0929 19:12:46.214549 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:46.214581 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260929 19:12:46.214943 25285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:46.214867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.2:34347)
0929 19:12:46.214911 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:46.214931 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260929 19:12:46.215050 25283 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 19:12:46.298640 25286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:12:46.298846 25286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:12:46.298869 25286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:12:46.299964 25286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:12:46.300002 25286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:12:46.300027 25286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:12:46.301446 25286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:46237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:40443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=42143
--tserver_master_addrs=127.9.236.62:46247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:12:46.301636 25286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:12:46.301817 25286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:12:46.307307 25286 server_base.cc:1034] running on GCE node
W20260929 19:12:46.307214 25292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:46.307215 25291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:12:46.307281 25294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:12:46.307690 25286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:12:46.307979 25286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:12:46.309165 25286 hybrid_clock.cc:648] HybridClock initialized: now 1790709166309144 us; error 38 us; skew 500 ppm
Sep 29 19:12:46 dist-test-slave-wftq krb5kdc[23850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709166, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:12:46.311717 25286 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:12:46.312158 25286 webserver.cc:460] Webserver started at http://127.9.236.3:42143/ using document root <none> and password file <none>
I20260929 19:12:46.312369 25286 fs_manager.cc:359] Metadata directory not provided
I20260929 19:12:46.312417 25286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:12:46.313822 25286 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:46.314479 25301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:46.314662 25286 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:12:46.314744 25286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "90886c62688a4d4c92ebf9f840f0b5d2"
format_stamp: "Formatted at 2026-09-29 19:12:44 on dist-test-slave-wftq"
server_key: "1d890a1c5f8135852767dbfcfb5238dc"
server_key_iv: "2f25a1e23de69651bdc13cc0b0ffcf95"
server_key_version: "encryptionkey@0"
I20260929 19:12:46.315101 25286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:12:46.328088 25286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:12:46.330073 25286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:12:46.330188 25286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:12:46.330415 25286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:12:46.330696 25286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:12:46.330731 25286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:46.330761 25286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:12:46.330789 25286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:12:46.337412 25286 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:40443
I20260929 19:12:46.337502 25414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:40443 every 8 connection(s)
I20260929 19:12:46.337831 25286 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:12:46.339087 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:46.338645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.3:47741)
0929 19:12:46.338856 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:46.338946 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":118,"threads_started":1}
W20260929 19:12:46.339531 25415 heartbeater.cc:644] Failed to heartbeat to 127.9.236.62:46247 (0 consecutive failures): Network error: Failed to ping master at 127.9.236.62:46247: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
I20260929 19:12:46.339634 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 25286
I20260929 19:12:46.339995 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:46.339836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.3:42421)
0929 19:12:46.339912 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:46.339961 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260929 19:12:46.340384 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:46.340291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.3:47343)
0929 19:12:46.340340 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:46.340365 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 19:12:46.340487 25415 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 19:12:47.146930 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:47.146613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.1:34881)
0929 19:12:47.146786 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:47.146846 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":62,"threads_started":1}
I20260929 19:12:47.215685 25420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:47.215375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.2:49771)
0929 19:12:47.215536 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:47.215598 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":60,"threads_started":1}
I20260929 19:12:47.341229 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:47.340871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.3:44183)
0929 19:12:47.341043 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:47.341142 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":77,"threads_started":1}
I20260929 19:12:47.819568 25422 raft_consensus.cc:482] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:12:47.819705 25422 raft_consensus.cc:504] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } } peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } }
I20260929 19:12:47.820029 25422 leader_election.cc:288] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90886c62688a4d4c92ebf9f840f0b5d2 (127.9.236.3:40443), ca5b39449c13474f98f601cb46c0ea0b (127.9.236.2:43261)
Sep 29 19:12:47 dist-test-slave-wftq krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709166, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.2@KRBTEST.COM
Sep 29 19:12:47 dist-test-slave-wftq krb5kdc[23850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709166, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.3@KRBTEST.COM
I20260929 19:12:47.831146 25424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:47.820397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:55053 (local address 127.9.236.3:40443)
0929 19:12:47.820557 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:47.820569 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:47.820613 (+ 44us) server_negotiation.cc:407] Connection header received
0929 19:12:47.821088 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:47.821090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:47.821169 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:47.821291 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:47.822406 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:47.823284 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:47.824398 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:47.824807 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:47.824914 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:47.827634 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:47.827637 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:47.827637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:47.827648 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:47.829522 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:47.830022 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:47.830025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:47.830026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:47.830108 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:47.830326 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:47.830327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:47.830328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:47.830559 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:47.830631 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:47.830911 (+ 280us) server_negotiation.cc:299] Negotiation successful
0929 19:12:47.831040 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
I20260929 19:12:47.831194 25426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:47.820724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:52117 (local address 127.9.236.2:43261)
0929 19:12:47.820858 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 19:12:47.820872 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 19:12:47.820895 (+ 23us) server_negotiation.cc:407] Connection header received
0929 19:12:47.821576 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:12:47.821580 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:12:47.821637 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:12:47.821800 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:12:47.822284 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:47.823259 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:47.824400 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:12:47.824812 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:12:47.824916 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:47.827367 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:12:47.827372 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:12:47.827373 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:12:47.827385 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:12:47.829508 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:47.830037 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:47.830041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:47.830042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:47.830138 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:12:47.830331 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:12:47.830332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:12:47.830333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:12:47.830579 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:12:47.830654 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 19:12:47.830909 (+ 255us) server_negotiation.cc:299] Negotiation successful
0929 19:12:47.831060 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":52,"threads_started":1}
I20260929 19:12:47.831189 25425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:47.820519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:43261 (local address 127.9.236.1:52117)
0929 19:12:47.820617 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:47.820624 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:47.820670 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:47.821801 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:47.821802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:47.821992 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:47.822157 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:47.822163 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:47.823366 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:47.823368 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:47.824306 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:47.824309 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:47.824905 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:47.824906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:47.825033 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:47.825644 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:47.825652 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:47.827223 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:47.829601 (+ 2378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:47.829607 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:47.829609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:47.829949 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:47.830198 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:47.830199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:47.830199 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:47.830275 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:47.830642 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:47.830643 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:47.830758 (+ 115us) client_negotiation.cc:769] Sending connection context
0929 19:12:47.830851 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 19:12:47.831058 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":63,"thread_start_us":55,"threads_started":1}
I20260929 19:12:47.831300 25423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:47.820273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:40443 (local address 127.9.236.1:55053)
0929 19:12:47.820422 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:47.820434 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:12:47.820506 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:12:47.821361 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:12:47.821364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:12:47.821817 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:12:47.822290 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:47.822294 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:47.823383 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:47.823383 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:47.824303 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:12:47.824305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:12:47.824896 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:12:47.824897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:12:47.825034 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:12:47.825690 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:12:47.825695 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:12:47.827531 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:12:47.829638 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:47.829639 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:47.829639 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:47.829950 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:47.830165 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:12:47.830166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:12:47.830166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:12:47.830271 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:12:47.830621 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:12:47.830628 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:12:47.830763 (+ 135us) client_negotiation.cc:769] Sending connection context
0929 19:12:47.830852 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 19:12:47.831149 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":122,"thread_start_us":65,"threads_started":1}
W20260929 19:12:47.832190 25035 leader_election.cc:341] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca5b39449c13474f98f601cb46c0ea0b (127.9.236.2:43261): Not found: Tablet not found: d5045d013792492d93ccf836c07a4622
W20260929 19:12:47.832572 25035 leader_election.cc:341] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90886c62688a4d4c92ebf9f840f0b5d2 (127.9.236.3:40443): Not found: Tablet not found: d5045d013792492d93ccf836c07a4622
I20260929 19:12:47.832653 25035 leader_election.cc:302] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0625621a9574adca65c1fa807eb83a8; no voters: 90886c62688a4d4c92ebf9f840f0b5d2, ca5b39449c13474f98f601cb46c0ea0b
I20260929 19:12:47.832806 25422 raft_consensus.cc:2726] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:12:48.147607 25425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:48.147415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.1:55961)
0929 19:12:48.147507 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:48.147552 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260929 19:12:48.216529 25427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:48.216230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.2:49831)
0929 19:12:48.216426 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:48.216484 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":99,"threads_started":1}
I20260929 19:12:48.342064 25428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:48.341780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.3:44453)
0929 19:12:48.341945 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:48.342005 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":68,"threads_started":1}
I20260929 19:12:49.148274 25429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:49.148053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.1:39771)
0929 19:12:49.148188 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:49.148224 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
I20260929 19:12:49.217295 25430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:49.217025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.2:45553)
0929 19:12:49.217208 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:49.217252 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":122,"threads_started":1}
I20260929 19:12:49.342868 25431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:49.342608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.3:60539)
0929 19:12:49.342751 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:49.342806 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":61,"threads_started":1}
I20260929 19:12:49.857383 25432 raft_consensus.cc:482] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:12:49.857482 25432 raft_consensus.cc:504] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } } peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } }
I20260929 19:12:49.857637 25432 leader_election.cc:288] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90886c62688a4d4c92ebf9f840f0b5d2 (127.9.236.3:40443), ca5b39449c13474f98f601cb46c0ea0b (127.9.236.2:43261)
W20260929 19:12:49.858134 25035 leader_election.cc:341] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90886c62688a4d4c92ebf9f840f0b5d2 (127.9.236.3:40443): Not found: Tablet not found: d5045d013792492d93ccf836c07a4622
W20260929 19:12:49.858213 25035 leader_election.cc:341] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca5b39449c13474f98f601cb46c0ea0b (127.9.236.2:43261): Not found: Tablet not found: d5045d013792492d93ccf836c07a4622
I20260929 19:12:49.858251 25035 leader_election.cc:302] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0625621a9574adca65c1fa807eb83a8; no voters: 90886c62688a4d4c92ebf9f840f0b5d2, ca5b39449c13474f98f601cb46c0ea0b
I20260929 19:12:49.858332 25432 raft_consensus.cc:2726] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:12:50.148965 25433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:50.148678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.1:38177)
0929 19:12:50.148849 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:50.148901 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":103,"threads_started":1}
I20260929 19:12:50.218116 25434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:50.217823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.2:41499)
0929 19:12:50.217988 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:50.218038 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":76,"threads_started":1}
I20260929 19:12:50.343731 25435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:50.343394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.3:49753)
0929 19:12:50.343603 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:50.343664 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":109,"threads_started":1}
I20260929 19:12:51.149730 25436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:51.149488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.1:40479)
0929 19:12:51.149638 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:51.149683 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":81,"threads_started":1}
I20260929 19:12:51.218839 25437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:51.218563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.2:34229)
0929 19:12:51.218709 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:51.218769 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":61,"threads_started":1}
I20260929 19:12:51.344624 25438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:51.344258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.3:52575)
0929 19:12:51.344490 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:51.344555 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":67,"threads_started":1}
I20260929 19:12:52.150513 25439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:52.150237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.1:42915)
0929 19:12:52.150392 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:52.150445 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":65,"threads_started":1}
I20260929 19:12:52.219655 25440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:52.219371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.2:34839)
0929 19:12:52.219531 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:52.219584 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":62,"threads_started":1}
I20260929 19:12:52.345705 25441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:52.345408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.3:40377)
0929 19:12:52.345574 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:52.345633 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":98,"threads_started":1}
I20260929 19:12:52.355566 25442 raft_consensus.cc:482] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:12:52.355707 25442 raft_consensus.cc:504] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } } peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } }
I20260929 19:12:52.355908 25442 leader_election.cc:288] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90886c62688a4d4c92ebf9f840f0b5d2 (127.9.236.3:40443), ca5b39449c13474f98f601cb46c0ea0b (127.9.236.2:43261)
W20260929 19:12:52.356426 25035 leader_election.cc:341] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90886c62688a4d4c92ebf9f840f0b5d2 (127.9.236.3:40443): Not found: Tablet not found: d5045d013792492d93ccf836c07a4622
W20260929 19:12:52.356510 25035 leader_election.cc:341] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca5b39449c13474f98f601cb46c0ea0b (127.9.236.2:43261): Not found: Tablet not found: d5045d013792492d93ccf836c07a4622
I20260929 19:12:52.356547 25035 leader_election.cc:302] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0625621a9574adca65c1fa807eb83a8; no voters: 90886c62688a4d4c92ebf9f840f0b5d2, ca5b39449c13474f98f601cb46c0ea0b
I20260929 19:12:52.356673 25442 raft_consensus.cc:2726] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:12:53.151302 25443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:53.150999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.1:35697)
0929 19:12:53.151195 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:53.151251 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":86,"threads_started":1}
I20260929 19:12:53.220463 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:53.220171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.2:36817)
0929 19:12:53.220348 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:53.220408 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":75,"threads_started":1}
I20260929 19:12:53.346513 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:53.346189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.3:49785)
0929 19:12:53.346355 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:53.346417 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":66,"threads_started":1}
I20260929 19:12:54.152238 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:54.151925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.1:42409)
0929 19:12:54.152119 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:54.152181 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":91,"threads_started":1}
I20260929 19:12:54.221138 25447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:54.220871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.2:43577)
0929 19:12:54.221018 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:54.221055 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":76,"threads_started":1}
I20260929 19:12:54.347448 25448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:54.347124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.3:43241)
0929 19:12:54.347285 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:54.347358 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":67,"threads_started":1}
I20260929 19:12:55.152992 25449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:55.152748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.1:47779)
0929 19:12:55.152890 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:55.152929 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":59,"threads_started":1}
I20260929 19:12:55.221909 25450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:55.221613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.2:35253)
0929 19:12:55.221778 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:55.221845 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":67,"threads_started":1}
I20260929 19:12:55.348152 25451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:55.347936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.3:44931)
0929 19:12:55.348067 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:55.348100 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260929 19:12:56.153743 25452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:56.153495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.1:35793)
0929 19:12:56.153655 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:56.153694 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":98,"threads_started":1}
I20260929 19:12:56.222697 25453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:56.222442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.2:60371)
0929 19:12:56.222595 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:56.222644 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260929 19:12:56.349144 25454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:56.348770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.3:45171)
0929 19:12:56.348943 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:56.349009 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":69,"threads_started":1}
I20260929 19:12:56.751201 25455 raft_consensus.cc:482] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:12:56.751317 25455 raft_consensus.cc:504] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } } peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } }
I20260929 19:12:56.751497 25455 leader_election.cc:288] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90886c62688a4d4c92ebf9f840f0b5d2 (127.9.236.3:40443), ca5b39449c13474f98f601cb46c0ea0b (127.9.236.2:43261)
W20260929 19:12:56.751986 25035 leader_election.cc:341] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90886c62688a4d4c92ebf9f840f0b5d2 (127.9.236.3:40443): Not found: Tablet not found: d5045d013792492d93ccf836c07a4622
W20260929 19:12:56.752086 25035 leader_election.cc:341] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca5b39449c13474f98f601cb46c0ea0b (127.9.236.2:43261): Not found: Tablet not found: d5045d013792492d93ccf836c07a4622
I20260929 19:12:56.752120 25035 leader_election.cc:302] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0625621a9574adca65c1fa807eb83a8; no voters: 90886c62688a4d4c92ebf9f840f0b5d2, ca5b39449c13474f98f601cb46c0ea0b
I20260929 19:12:56.752233 25455 raft_consensus.cc:2726] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:12:57.154579 25456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:57.154251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.1:59825)
0929 19:12:57.154451 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:57.154514 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":94,"threads_started":1}
I20260929 19:12:57.223484 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:57.223259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.2:51177)
0929 19:12:57.223401 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:57.223440 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
I20260929 19:12:57.350011 25458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:57.349710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.3:60385)
0929 19:12:57.349883 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:57.349960 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":84,"threads_started":1}
I20260929 19:12:58.155437 25459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:58.155146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.1:48049)
0929 19:12:58.155323 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:58.155377 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":86,"threads_started":1}
I20260929 19:12:58.224253 25460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:58.223968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.2:53491)
0929 19:12:58.224141 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:58.224200 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":76,"threads_started":1}
I20260929 19:12:58.350798 25461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:58.350538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.3:60327)
0929 19:12:58.350692 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:58.350727 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":78,"threads_started":1}
I20260929 19:12:59.156237 25462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:59.155939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.1:44915)
0929 19:12:59.156129 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:59.156187 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":87,"threads_started":1}
I20260929 19:12:59.225052 25463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:59.224753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.2:52817)
0929 19:12:59.224943 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:59.225001 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":97,"threads_started":1}
I20260929 19:12:59.230120 25464 raft_consensus.cc:482] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:12:59.230266 25464 raft_consensus.cc:504] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90886c62688a4d4c92ebf9f840f0b5d2" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40443 } } peers { permanent_uuid: "ca5b39449c13474f98f601cb46c0ea0b" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43261 } } peers { permanent_uuid: "b0625621a9574adca65c1fa807eb83a8" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 42881 } }
I20260929 19:12:59.230521 25464 leader_election.cc:288] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90886c62688a4d4c92ebf9f840f0b5d2 (127.9.236.3:40443), ca5b39449c13474f98f601cb46c0ea0b (127.9.236.2:43261)
W20260929 19:12:59.230994 25035 leader_election.cc:341] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca5b39449c13474f98f601cb46c0ea0b (127.9.236.2:43261): Not found: Tablet not found: d5045d013792492d93ccf836c07a4622
W20260929 19:12:59.231266 25035 leader_election.cc:341] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90886c62688a4d4c92ebf9f840f0b5d2 (127.9.236.3:40443): Not found: Tablet not found: d5045d013792492d93ccf836c07a4622
I20260929 19:12:59.231328 25035 leader_election.cc:302] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0625621a9574adca65c1fa807eb83a8; no voters: 90886c62688a4d4c92ebf9f840f0b5d2, ca5b39449c13474f98f601cb46c0ea0b
I20260929 19:12:59.231427 25464 raft_consensus.cc:2726] T d5045d013792492d93ccf836c07a4622 P b0625621a9574adca65c1fa807eb83a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:12:59.351691 25465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:12:59.351394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.3:40561)
0929 19:12:59.351568 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0929 19:12:59.351623 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":75,"threads_started":1}
I20260929 19:13:00.157048 25466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:00.156730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.1:40889)
0929 19:13:00.156924 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:00.156982 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":83,"threads_started":1}
I20260929 19:13:00.225854 25467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:00.225605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.2:53159)
0929 19:13:00.225760 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:00.225807 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
I20260929 19:13:00.352640 25468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:00.352340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.3:41667)
0929 19:13:00.352532 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:00.352588 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":95,"threads_started":1}
I20260929 19:13:01.157982 25469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:01.157721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.1:43001)
0929 19:13:01.157873 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:01.157920 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
I20260929 19:13:01.226585 25470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:01.226354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.2:34833)
0929 19:13:01.226500 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:01.226539 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":82,"threads_started":1}
I20260929 19:13:01.347729 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 24990
I20260929 19:13:01.353070 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 25154
I20260929 19:13:01.354648 25471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:01.353225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46247 (local address 127.9.236.3:44393)
0929 19:13:01.354556 (+ 1331us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:01.354593 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:46247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1310,"thread_start_us":81,"threads_started":1}
I20260929 19:13:01.357656 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 25286
2026-09-29T19:13:01Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18103 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:13:01 dist-test-slave-wftq krb5kdc[25474](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:13:01 dist-test-slave-wftq krb5kdc[25474](info): set up 2 sockets
Sep 29 19:13:01 dist-test-slave-wftq krb5kdc[25474](info): commencing operation
krb5kdc: starting...
W20260929 19:13:02.301169 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.924s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:13:02 dist-test-slave-wftq krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:13:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:13:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:13:02.420306 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:37539
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:37539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:13:02.507042 25487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:13:02.507277 25487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:13:02.507298 25487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:13:02.508476 25487 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 19:13:02.508510 25487 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 19:13:02.508522 25487 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 19:13:02.508533 25487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:13:02.508543 25487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:13:02.508554 25487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:13:02.508564 25487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:13:02.508575 25487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:13:02.510104 25487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:37539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:37539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:13:02.510316 25487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:13:02.510530 25487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:13:02.516104 25492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:13:02.516126 25495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:13:02.516494 25487 server_base.cc:1034] running on GCE node
W20260929 19:13:02.517575 25493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:13:02.518056 25487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:13:02.518402 25487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:13:02.519528 25487 hybrid_clock.cc:648] HybridClock initialized: now 1790709182519508 us; error 43 us; skew 500 ppm
Sep 29 19:13:02 dist-test-slave-wftq krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709182, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:13:02.522781 25487 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:13:02.523310 25487 webserver.cc:460] Webserver started at http://127.9.236.62:40689/ using document root <none> and password file <none>
I20260929 19:13:02.523568 25487 fs_manager.cc:359] Metadata directory not provided
I20260929 19:13:02.523619 25487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:13:02.523748 25487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:13:02.524932 25487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "b9850d4c2a4043cd86865de232d53069"
format_stamp: "Formatted at 2026-09-29 19:13:02 on dist-test-slave-wftq"
server_key: "acd5f8dacf21040a545707bdda8da682"
server_key_iv: "6a43419585487c1426b34cf37f36b63d"
server_key_version: "encryptionkey@0"
I20260929 19:13:02.525313 25487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "b9850d4c2a4043cd86865de232d53069"
format_stamp: "Formatted at 2026-09-29 19:13:02 on dist-test-slave-wftq"
server_key: "acd5f8dacf21040a545707bdda8da682"
server_key_iv: "6a43419585487c1426b34cf37f36b63d"
server_key_version: "encryptionkey@0"
I20260929 19:13:02.526664 25487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:13:02.527410 25502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:02.527607 25487 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:13:02.527696 25487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "b9850d4c2a4043cd86865de232d53069"
format_stamp: "Formatted at 2026-09-29 19:13:02 on dist-test-slave-wftq"
server_key: "acd5f8dacf21040a545707bdda8da682"
server_key_iv: "6a43419585487c1426b34cf37f36b63d"
server_key_version: "encryptionkey@0"
I20260929 19:13:02.527760 25487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:13:02.547238 25487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:13:02.549300 25487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:13:02.549481 25487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:13:02.554095 25487 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:37539
I20260929 19:13:02.554141 25554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:37539 every 8 connection(s)
I20260929 19:13:02.554508 25487 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:13:02.555454 25555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:13:02.555768 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 25487
I20260929 19:13:02.555868 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:13:02.556094 10160 external_mini_cluster.cc:1428] Setting key 86ffd2f0e50b2e207e7d2d97f0a78ca8
I20260929 19:13:02.558248 25555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069: Bootstrap starting.
I20260929 19:13:02.558840 25555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069: Neither blocks nor log segments found. Creating new log.
I20260929 19:13:02.559130 25555 log.cc:824] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069: Log is configured to *not* fsync() on all Append() calls
I20260929 19:13:02.559738 25555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069: No bootstrap required, opened a new log
I20260929 19:13:02.561697 25555 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9850d4c2a4043cd86865de232d53069" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37539 } }
I20260929 19:13:02.561844 25555 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:13:02.561882 25555 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9850d4c2a4043cd86865de232d53069, State: Initialized, Role: FOLLOWER
I20260929 19:13:02.562007 25555 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9850d4c2a4043cd86865de232d53069" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37539 } }
I20260929 19:13:02.562080 25555 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:13:02.562111 25555 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:13:02.562157 25555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:13:02.562834 25555 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9850d4c2a4043cd86865de232d53069" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37539 } }
Sep 29 19:13:02 dist-test-slave-wftq krb5kdc[25474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:13:02.562949 25555 leader_election.cc:302] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9850d4c2a4043cd86865de232d53069; no voters:
I20260929 19:13:02.563118 25555 leader_election.cc:288] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:13:02.563194 25560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:13:02.563393 25555 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:13:02.563431 25560 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 1 LEADER]: Becoming Leader. State: Replica: b9850d4c2a4043cd86865de232d53069, State: Running, Role: LEADER
I20260929 19:13:02.563621 25560 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9850d4c2a4043cd86865de232d53069" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37539 } }
I20260929 19:13:02.564536 25561 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9850d4c2a4043cd86865de232d53069" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9850d4c2a4043cd86865de232d53069" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37539 } } }
I20260929 19:13:02.564648 25561 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [sys.catalog]: This master's current role is: LEADER
I20260929 19:13:02.564911 25562 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9850d4c2a4043cd86865de232d53069. Latest consensus state: current_term: 1 leader_uuid: "b9850d4c2a4043cd86865de232d53069" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9850d4c2a4043cd86865de232d53069" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37539 } } }
I20260929 19:13:02.565162 25562 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [sys.catalog]: This master's current role is: LEADER
W20260929 19:13:02.565685 25575 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:13:02.565788 25575 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:13:02.565894 25572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:13:02.566097 25572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:13:02.567361 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:02.556645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42992 (local address 127.9.236.62:37539)
0929 19:13:02.556875 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:02.556884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:02.556914 (+ 30us) server_negotiation.cc:407] Connection header received
0929 19:13:02.557505 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:02.557513 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:02.557567 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:02.557709 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:13:02.558341 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:02.559494 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:02.560241 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:02.560690 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:02.560796 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:02.563488 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:13:02.563493 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:13:02.563493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:13:02.563506 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:13:02.565538 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:02.565923 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:02.565928 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:02.565928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:02.566077 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:02.566354 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:02.566355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:02.566356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:02.566688 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:13:02.566860 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:02.566997 (+ 137us) server_negotiation.cc:299] Negotiation successful
0929 19:13:02.567161 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":108,"threads_started":1}
I20260929 19:13:02.567996 25572 catalog_manager.cc:1347] Generated new cluster ID: 3a30a93c8b4045a6a4616894898973c9
I20260929 19:13:02.568048 25572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:13:02.582769 25572 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:13:02.583600 25572 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:13:02.589745 25572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069: Generated new TSK 0
I20260929 19:13:02.589960 25572 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.236.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
I20260929 19:13:02.620333 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:37539
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:13:02.704957 25581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:13:02.705222 25581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:13:02.705250 25581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:13:02.706413 25581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:13:02.706461 25581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:13:02.706486 25581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:13:02.707931 25581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:37539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:13:02.708186 25581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:13:02.708419 25581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:13:02.713907 25587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:13:02.713946 25586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:13:02.713946 25581 server_base.cc:1034] running on GCE node
W20260929 19:13:02.713902 25589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:13:02.714398 25581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:13:02.714666 25581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:13:02.715814 25581 hybrid_clock.cc:648] HybridClock initialized: now 1790709182715813 us; error 49 us; skew 500 ppm
Sep 29 19:13:02 dist-test-slave-wftq krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709182, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:13:02.718236 25581 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:13:02.718684 25581 webserver.cc:460] Webserver started at http://127.9.236.1:43677/ using document root <none> and password file <none>
I20260929 19:13:02.718899 25581 fs_manager.cc:359] Metadata directory not provided
I20260929 19:13:02.718945 25581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:13:02.719069 25581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:13:02.720049 25581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb"
format_stamp: "Formatted at 2026-09-29 19:13:02 on dist-test-slave-wftq"
server_key: "16eedb70479a5c26aa6127febe4b36dc"
server_key_iv: "09c088a63334f424604990ff1da0f2d1"
server_key_version: "encryptionkey@0"
I20260929 19:13:02.720362 25581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb"
format_stamp: "Formatted at 2026-09-29 19:13:02 on dist-test-slave-wftq"
server_key: "16eedb70479a5c26aa6127febe4b36dc"
server_key_iv: "09c088a63334f424604990ff1da0f2d1"
server_key_version: "encryptionkey@0"
I20260929 19:13:02.721700 25581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:13:02.722534 25596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:02.722782 25581 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:13:02.722872 25581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb"
format_stamp: "Formatted at 2026-09-29 19:13:02 on dist-test-slave-wftq"
server_key: "16eedb70479a5c26aa6127febe4b36dc"
server_key_iv: "09c088a63334f424604990ff1da0f2d1"
server_key_version: "encryptionkey@0"
I20260929 19:13:02.722932 25581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:13:02.734721 25581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:13:02.736691 25581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:13:02.736821 25581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:13:02.737030 25581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:13:02.737340 25581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:13:02.737373 25581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:02.737401 25581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:13:02.737418 25581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:02.743775 25581 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:41083
I20260929 19:13:02.743846 25709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:41083 every 8 connection(s)
I20260929 19:13:02.744145 25581 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:13:02.745357 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 25581
I20260929 19:13:02.745492 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:13:02.745709 10160 external_mini_cluster.cc:1428] Setting key 3cc4f15a6db0760c804b0dd494611cf6
Sep 29 19:13:02 dist-test-slave-wftq krb5kdc[25474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709182, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:13:02.755044 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:02.744944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:57709 (local address 127.9.236.62:37539)
0929 19:13:02.745043 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:02.745045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:02.745286 (+ 241us) server_negotiation.cc:407] Connection header received
0929 19:13:02.746059 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:02.746061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:02.746076 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:02.746130 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:13:02.747468 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:02.747997 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:02.749076 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:02.749405 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:02.749468 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:02.752060 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:13:02.752062 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:13:02.752063 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:13:02.752072 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:13:02.753672 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:02.754056 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:02.754057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:02.754057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:02.754123 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:02.754356 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:02.754356 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:02.754357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:02.754502 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:13:02.754580 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:02.754827 (+ 247us) server_negotiation.cc:299] Negotiation successful
0929 19:13:02.754935 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 19:13:02.755069 25712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:02.744879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.1:57709)
0929 19:13:02.745060 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:02.745148 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0929 19:13:02.745942 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:13:02.746221 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:13:02.746226 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:13:02.747000 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:13:02.747332 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:02.747341 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:02.748126 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:02.748128 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:02.748978 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:02.748981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:02.749494 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:02.749495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:02.749663 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:02.750234 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:13:02.750245 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:13:02.751952 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:13:02.753753 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:13:02.753759 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:13:02.753761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:13:02.753974 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:13:02.754192 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:13:02.754193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:13:02.754193 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:13:02.754292 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:13:02.754572 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:13:02.754575 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:13:02.754694 (+ 119us) client_negotiation.cc:769] Sending connection context
0929 19:13:02.754776 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 19:13:02.754900 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for kudu/127.9.236.2@KRBTEST.COM; defaulting to no policy
I20260929 19:13:02.755817 25710 heartbeater.cc:347] Connected to a master server at 127.9.236.62:37539
I20260929 19:13:02.755961 25710 heartbeater.cc:464] Registering TS with master...
I20260929 19:13:02.756188 25710 heartbeater.cc:511] Master 127.9.236.62:37539 requested a full tablet report, sending...
I20260929 19:13:02.756753 25519 ts_manager.cc:194] Registered new tserver with Master: fc1d458bd6ff4dd39b4cb0303b7c4dbb (127.9.236.1:41083)
I20260929 19:13:02.757424 25519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:57709
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
I20260929 19:13:02.773368 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:37539
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:13:02.860319 25715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:13:02.860548 25715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:13:02.860570 25715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:13:02.861747 25715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:13:02.861794 25715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:13:02.861819 25715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:13:02.863281 25715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:37539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:13:02.863477 25715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:13:02.863724 25715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:13:02.869298 25721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:13:02.869333 25723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:13:02.869333 25720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:13:02.869359 25715 server_base.cc:1034] running on GCE node
I20260929 19:13:02.869745 25715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:13:02.869980 25715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:13:02.871140 25715 hybrid_clock.cc:648] HybridClock initialized: now 1790709182871119 us; error 42 us; skew 500 ppm
Sep 29 19:13:02 dist-test-slave-wftq krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709182, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:13:02.874127 25715 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:13:02.874723 25715 webserver.cc:460] Webserver started at http://127.9.236.2:34799/ using document root <none> and password file <none>
I20260929 19:13:02.874955 25715 fs_manager.cc:359] Metadata directory not provided
I20260929 19:13:02.875007 25715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:13:02.875147 25715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:13:02.876335 25715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "f8375ab687994ea6868fd4b1d054b00e"
format_stamp: "Formatted at 2026-09-29 19:13:02 on dist-test-slave-wftq"
server_key: "ce9fc6758ea6e61a1be921381b9ee3de"
server_key_iv: "6ef94e67ffc09c1d95234a67f774743d"
server_key_version: "encryptionkey@0"
I20260929 19:13:02.876713 25715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "f8375ab687994ea6868fd4b1d054b00e"
format_stamp: "Formatted at 2026-09-29 19:13:02 on dist-test-slave-wftq"
server_key: "ce9fc6758ea6e61a1be921381b9ee3de"
server_key_iv: "6ef94e67ffc09c1d95234a67f774743d"
server_key_version: "encryptionkey@0"
I20260929 19:13:02.878305 25715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:13:02.879191 25730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:02.879437 25715 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:13:02.879534 25715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "f8375ab687994ea6868fd4b1d054b00e"
format_stamp: "Formatted at 2026-09-29 19:13:02 on dist-test-slave-wftq"
server_key: "ce9fc6758ea6e61a1be921381b9ee3de"
server_key_iv: "6ef94e67ffc09c1d95234a67f774743d"
server_key_version: "encryptionkey@0"
I20260929 19:13:02.879613 25715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:13:02.896634 25715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:13:02.898910 25715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:13:02.899053 25715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:13:02.899313 25715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:13:02.899668 25715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:13:02.899710 25715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:02.899739 25715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:13:02.899752 25715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:02.907146 25715 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:42955
I20260929 19:13:02.907210 25843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:42955 every 8 connection(s)
I20260929 19:13:02.907558 25715 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:13:02.909155 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 25715
I20260929 19:13:02.909268 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:13:02.909514 10160 external_mini_cluster.cc:1428] Setting key e4b5ec5fa48ccc3031c30b1231b4c9f4
Sep 29 19:13:02 dist-test-slave-wftq krb5kdc[25474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709182, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.236.3@KRBTEST.COM; defaulting to no policy
I20260929 19:13:02.918829 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:02.908428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:55465 (local address 127.9.236.62:37539)
0929 19:13:02.908491 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:02.908493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:02.908715 (+ 222us) server_negotiation.cc:407] Connection header received
0929 19:13:02.909485 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:02.909486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:02.909496 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:02.909542 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:13:02.910826 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:02.911334 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:02.912623 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:02.912912 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:02.912986 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:02.915683 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:13:02.915685 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:13:02.915685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:13:02.915693 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:13:02.917387 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:02.917837 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:02.917838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:02.917839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:02.917891 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:02.918138 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:02.918144 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:02.918144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:02.918286 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:13:02.918359 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:02.918668 (+ 309us) server_negotiation.cc:299] Negotiation successful
0929 19:13:02.918743 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 19:13:02.919102 25846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:02.908384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.2:55465)
0929 19:13:02.908571 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:02.908617 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 19:13:02.909346 (+ 729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:13:02.909653 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:13:02.909657 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:13:02.910354 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:13:02.910718 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:02.910726 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:02.911432 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:02.911433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:02.912534 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:02.912538 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:02.913012 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:02.913013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:02.913179 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:02.913809 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:13:02.913817 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:13:02.915576 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:13:02.917485 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:13:02.917492 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:13:02.917495 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:13:02.917750 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:13:02.917966 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:13:02.917966 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:13:02.917966 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:13:02.918067 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:13:02.918367 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:13:02.918373 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:13:02.918525 (+ 152us) client_negotiation.cc:769] Sending connection context
0929 19:13:02.918639 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 19:13:02.918792 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":94,"threads_started":1}
I20260929 19:13:02.919736 25844 heartbeater.cc:347] Connected to a master server at 127.9.236.62:37539
I20260929 19:13:02.919859 25844 heartbeater.cc:464] Registering TS with master...
I20260929 19:13:02.920084 25844 heartbeater.cc:511] Master 127.9.236.62:37539 requested a full tablet report, sending...
I20260929 19:13:02.920625 25519 ts_manager.cc:194] Registered new tserver with Master: f8375ab687994ea6868fd4b1d054b00e (127.9.236.2:42955)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.3@KRBTEST.COM" created.
I20260929 19:13:02.921129 25519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:55465
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
I20260929 19:13:02.935724 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:37539
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:13:03.018397 25849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:13:03.018607 25849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:13:03.018638 25849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:13:03.019765 25849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:13:03.019807 25849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:13:03.019842 25849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:13:03.021278 25849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:37539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:13:03.021507 25849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:13:03.021708 25849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:13:03.026855 25854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:13:03.026914 25857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:13:03.026928 25855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:13:03.026944 25849 server_base.cc:1034] running on GCE node
I20260929 19:13:03.027325 25849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:13:03.027563 25849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:13:03.028733 25849 hybrid_clock.cc:648] HybridClock initialized: now 1790709183028706 us; error 37 us; skew 500 ppm
Sep 29 19:13:03 dist-test-slave-wftq krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709183, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:13:03.031842 25849 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:13:03.032328 25849 webserver.cc:460] Webserver started at http://127.9.236.3:42627/ using document root <none> and password file <none>
I20260929 19:13:03.032547 25849 fs_manager.cc:359] Metadata directory not provided
I20260929 19:13:03.032596 25849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:13:03.032739 25849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:13:03.033990 25849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d"
format_stamp: "Formatted at 2026-09-29 19:13:03 on dist-test-slave-wftq"
server_key: "0322d575d2c7ea48b37db87f083be7d1"
server_key_iv: "59a8347e8ad657f3f6378aa4fd70b23f"
server_key_version: "encryptionkey@0"
I20260929 19:13:03.034396 25849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d"
format_stamp: "Formatted at 2026-09-29 19:13:03 on dist-test-slave-wftq"
server_key: "0322d575d2c7ea48b37db87f083be7d1"
server_key_iv: "59a8347e8ad657f3f6378aa4fd70b23f"
server_key_version: "encryptionkey@0"
I20260929 19:13:03.035777 25849 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:13:03.036576 25864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:03.036805 25849 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:13:03.036896 25849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d"
format_stamp: "Formatted at 2026-09-29 19:13:03 on dist-test-slave-wftq"
server_key: "0322d575d2c7ea48b37db87f083be7d1"
server_key_iv: "59a8347e8ad657f3f6378aa4fd70b23f"
server_key_version: "encryptionkey@0"
I20260929 19:13:03.036965 25849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:13:03.059770 25849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:13:03.062124 25849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:13:03.062275 25849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:13:03.062542 25849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:13:03.062877 25849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:13:03.062911 25849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:03.062944 25849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:13:03.062968 25849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:03.070344 25849 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:35887
I20260929 19:13:03.070437 25977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:35887 every 8 connection(s)
I20260929 19:13:03.070760 25849 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:13:03.071269 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 25849
I20260929 19:13:03.071359 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:13:03.071557 10160 external_mini_cluster.cc:1428] Setting key 2908ff5ff8edc062995792552211cdfb
Sep 29 19:13:03 dist-test-slave-wftq krb5kdc[25474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709183, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:13:03.080490 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:03.071627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:49717 (local address 127.9.236.62:37539)
0929 19:13:03.071689 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:03.071691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:03.071919 (+ 228us) server_negotiation.cc:407] Connection header received
0929 19:13:03.072476 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:03.072476 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:03.072486 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:03.072541 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:13:03.073601 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.074106 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.075136 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.075327 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.075387 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:03.077911 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:13:03.077912 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:13:03.077913 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:13:03.077919 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:13:03.079212 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:03.079623 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:03.079624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:03.079624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:03.079674 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:03.079884 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:03.079885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:03.079885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:03.079993 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:13:03.080045 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:03.080325 (+ 280us) server_negotiation.cc:299] Negotiation successful
0929 19:13:03.080398 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 19:13:03.080708 25980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:03.071596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.3:49717)
0929 19:13:03.071773 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:03.071830 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0929 19:13:03.072383 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:13:03.072604 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:13:03.072608 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:13:03.073275 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:13:03.073507 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:03.073512 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.074187 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.074187 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:03.075059 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:03.075062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.075448 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.075449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:03.075580 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:03.076149 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:13:03.076159 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:13:03.077822 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:13:03.079308 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:13:03.079315 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:13:03.079318 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:13:03.079544 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:13:03.079729 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:13:03.079730 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:13:03.079730 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:13:03.079832 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:13:03.080044 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:13:03.080050 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:13:03.080205 (+ 155us) client_negotiation.cc:769] Sending connection context
0929 19:13:03.080295 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 19:13:03.080472 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260929 19:13:03.081270 25978 heartbeater.cc:347] Connected to a master server at 127.9.236.62:37539
I20260929 19:13:03.081359 25978 heartbeater.cc:464] Registering TS with master...
I20260929 19:13:03.081534 25978 heartbeater.cc:511] Master 127.9.236.62:37539 requested a full tablet report, sending...
I20260929 19:13:03.081951 25519 ts_manager.cc:194] Registered new tserver with Master: d0be3da1d3f24a4ab5a5c5dd9ac1b53d (127.9.236.3:35887)
I20260929 19:13:03.082300 25519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:49717
I20260929 19:13:03.083129 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:13:03.090197 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:03.084504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43004 (local address 127.9.236.62:37539)
0929 19:13:03.084563 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:03.084565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:03.084576 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:13:03.084624 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:03.084624 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:03.084636 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:03.084687 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:13:03.085194 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.085717 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.086464 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.086653 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.086717 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:03.087731 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:13:03.087732 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:13:03.087732 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:13:03.087739 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:13:03.089064 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:03.089395 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:03.089396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:03.089396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:03.089448 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:03.089632 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:03.089633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:03.089633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:03.089858 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:13:03.089958 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:03.090046 (+ 88us) server_negotiation.cc:299] Negotiation successful
0929 19:13:03.090111 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 19:13:03.091674 25519 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43004:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:13:03.093308 25519 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:13:03.100303 25992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:03.095980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54900 (local address 127.9.236.2:42955)
0929 19:13:03.096234 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:03.096240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:03.096255 (+ 15us) server_negotiation.cc:407] Connection header received
0929 19:13:03.096307 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:03.096310 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:03.096330 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:03.096437 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:13:03.096852 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.097988 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.099232 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.099871 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.099949 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:03.100006 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:03.100175 (+ 169us) server_negotiation.cc:299] Negotiation successful
0929 19:13:03.100219 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":103,"threads_started":1}
I20260929 19:13:03.100358 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:03.095818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:42955 (local address 127.0.0.1:54900)
0929 19:13:03.095995 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:03.096031 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 19:13:03.096134 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:13:03.096692 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:13:03.096693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:13:03.096695 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:13:03.096740 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:03.096742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.098073 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.098074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:03.099156 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:03.099159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.099969 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.099970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:03.100055 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:03.100069 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 19:13:03.100141 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 19:13:03.100209 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":70,"threads_started":1}
I20260929 19:13:03.100739 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:03.096077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53732 (local address 127.9.236.1:41083)
0929 19:13:03.096255 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:03.096259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:03.096448 (+ 189us) server_negotiation.cc:407] Connection header received
0929 19:13:03.096484 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:03.096485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:03.096498 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:03.096572 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:13:03.096771 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.097915 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.099445 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.100102 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.100169 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:03.100219 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:03.100574 (+ 355us) server_negotiation.cc:299] Negotiation successful
0929 19:13:03.100638 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":78,"threads_started":1}
I20260929 19:13:03.100942 25990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:03.095843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:41083 (local address 127.0.0.1:53732)
0929 19:13:03.096297 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:03.096305 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 19:13:03.096358 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:13:03.096598 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:13:03.096612 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:13:03.096615 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:13:03.096696 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:03.096699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.098019 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.098020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:03.099352 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:03.099354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.100193 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.100194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:03.100249 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:03.100253 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:13:03.100315 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 19:13:03.100879 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":48,"thread_start_us":68,"threads_started":1}
I20260929 19:13:03.101135 25778 tablet_service.cc:1467] Processing CreateTablet for tablet 22b7c3180cca4bfc9f0150237e65c76e (DEFAULT_TABLE table=test-table [id=411fc4e732a546e7af6d06a50846e05f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:13:03.101508 25778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22b7c3180cca4bfc9f0150237e65c76e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:13:03.101643 25991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:03.095858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:35887 (local address 127.0.0.1:56046)
0929 19:13:03.096553 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:03.096564 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:13:03.096624 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:13:03.096933 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:13:03.096933 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:13:03.096935 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:13:03.096991 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:03.096993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.097924 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.097925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:03.099349 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:03.099353 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.101407 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.101408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:03.101467 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:03.101471 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:13:03.101526 (+ 55us) client_negotiation.cc:240] Negotiation successful
0929 19:13:03.101570 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":29,"thread_start_us":100,"threads_started":1}
I20260929 19:13:03.101658 25994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:03.096108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56046 (local address 127.9.236.3:35887)
0929 19:13:03.096385 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:03.096390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:03.096789 (+ 399us) server_negotiation.cc:407] Connection header received
0929 19:13:03.096838 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:03.096840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:03.096856 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:03.096930 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:13:03.097047 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.097822 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.100267 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.101316 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.101394 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:03.101445 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:03.101542 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 19:13:03.101583 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":85,"threads_started":1}
I20260929 19:13:03.102475 25912 tablet_service.cc:1467] Processing CreateTablet for tablet 22b7c3180cca4bfc9f0150237e65c76e (DEFAULT_TABLE table=test-table [id=411fc4e732a546e7af6d06a50846e05f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:13:03.102807 25912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22b7c3180cca4bfc9f0150237e65c76e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:13:03.104049 25995 tablet_bootstrap.cc:492] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e: Bootstrap starting.
I20260929 19:13:03.104720 25995 tablet_bootstrap.cc:654] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e: Neither blocks nor log segments found. Creating new log.
I20260929 19:13:03.104856 25644 tablet_service.cc:1467] Processing CreateTablet for tablet 22b7c3180cca4bfc9f0150237e65c76e (DEFAULT_TABLE table=test-table [id=411fc4e732a546e7af6d06a50846e05f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:13:03.105198 25644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22b7c3180cca4bfc9f0150237e65c76e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:13:03.105268 25995 log.cc:824] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e: Log is configured to *not* fsync() on all Append() calls
I20260929 19:13:03.105897 25995 tablet_bootstrap.cc:492] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e: No bootstrap required, opened a new log
I20260929 19:13:03.105911 25997 tablet_bootstrap.cc:492] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: Bootstrap starting.
I20260929 19:13:03.105955 25995 ts_tablet_manager.cc:1397] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:13:03.106539 25997 tablet_bootstrap.cc:654] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: Neither blocks nor log segments found. Creating new log.
I20260929 19:13:03.107033 25997 log.cc:824] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: Log is configured to *not* fsync() on all Append() calls
I20260929 19:13:03.107853 25997 tablet_bootstrap.cc:492] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: No bootstrap required, opened a new log
I20260929 19:13:03.107924 25997 ts_tablet_manager.cc:1397] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:13:03.107859 25995 raft_consensus.cc:348] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:03.107987 25995 raft_consensus.cc:374] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:13:03.108009 25995 raft_consensus.cc:729] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8375ab687994ea6868fd4b1d054b00e, State: Initialized, Role: FOLLOWER
I20260929 19:13:03.108105 25995 consensus_queue.cc:260] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:03.108358 25995 ts_tablet_manager.cc:1428] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 19:13:03.108476 25844 heartbeater.cc:503] Master 127.9.236.62:37539 was elected leader, sending a full tablet report...
I20260929 19:13:03.108486 25999 tablet_bootstrap.cc:492] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Bootstrap starting.
I20260929 19:13:03.109368 25999 tablet_bootstrap.cc:654] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Neither blocks nor log segments found. Creating new log.
I20260929 19:13:03.109891 25999 log.cc:824] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Log is configured to *not* fsync() on all Append() calls
I20260929 19:13:03.109823 25997 raft_consensus.cc:348] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:03.110015 25997 raft_consensus.cc:374] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:13:03.110060 25997 raft_consensus.cc:729] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0be3da1d3f24a4ab5a5c5dd9ac1b53d, State: Initialized, Role: FOLLOWER
I20260929 19:13:03.110195 25997 consensus_queue.cc:260] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:03.110505 25997 ts_tablet_manager.cc:1428] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:13:03.110563 25978 heartbeater.cc:503] Master 127.9.236.62:37539 was elected leader, sending a full tablet report...
I20260929 19:13:03.110720 25999 tablet_bootstrap.cc:492] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb: No bootstrap required, opened a new log
I20260929 19:13:03.110783 25999 ts_tablet_manager.cc:1397] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 19:13:03.112562 25999 raft_consensus.cc:348] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:03.112711 25999 raft_consensus.cc:374] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:13:03.112741 25999 raft_consensus.cc:729] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1d458bd6ff4dd39b4cb0303b7c4dbb, State: Initialized, Role: FOLLOWER
I20260929 19:13:03.112850 25999 consensus_queue.cc:260] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:03.113134 25999 ts_tablet_manager.cc:1428] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 19:13:03.113206 25710 heartbeater.cc:503] Master 127.9.236.62:37539 was elected leader, sending a full tablet report...
W20260929 19:13:03.158216 25845 tablet.cc:2369] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:13:03.239853 26002 raft_consensus.cc:482] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:13:03.239980 26002 raft_consensus.cc:504] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:03.240358 26002 leader_election.cc:288] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8375ab687994ea6868fd4b1d054b00e (127.9.236.2:42955), fc1d458bd6ff4dd39b4cb0303b7c4dbb (127.9.236.1:41083)
I20260929 19:13:03.243805 25992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:03.240689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:36277 (local address 127.9.236.2:42955)
0929 19:13:03.240753 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:03.240755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:03.241027 (+ 272us) server_negotiation.cc:407] Connection header received
0929 19:13:03.241085 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:03.241086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:03.241157 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:03.241207 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:13:03.241404 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.242079 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.243108 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.243490 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.243549 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:03.243561 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:03.243700 (+ 139us) server_negotiation.cc:299] Negotiation successful
0929 19:13:03.243737 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 19:13:03.243822 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:03.240641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:42955 (local address 127.9.236.3:36277)
0929 19:13:03.240935 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:03.240948 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:13:03.241020 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:13:03.241224 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:13:03.241225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:13:03.241227 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:13:03.241291 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:03.241292 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.242197 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.242198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:03.243022 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:03.243024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.243562 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.243563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:03.243637 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:03.243641 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:13:03.243686 (+ 45us) client_negotiation.cc:240] Negotiation successful
0929 19:13:03.243737 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":93,"thread_start_us":125,"threads_started":1}
I20260929 19:13:03.243940 25980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:03.240641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:41083 (local address 127.9.236.3:59639)
0929 19:13:03.240703 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:03.240717 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:13:03.240767 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:13:03.241002 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:13:03.241003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:13:03.241006 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:13:03.241131 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:03.241134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.242093 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.242095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:03.242995 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:03.242998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.243635 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.243636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:03.243728 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:03.243732 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:13:03.243794 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 19:13:03.243849 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260929 19:13:03.243940 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:03.240696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:59639 (local address 127.9.236.1:41083)
0929 19:13:03.240768 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:03.240770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:03.240782 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:13:03.240864 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:03.240865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:03.240878 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:03.240974 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:13:03.241199 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.241980 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.243128 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.243553 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.243629 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:03.243644 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:03.243802 (+ 158us) server_negotiation.cc:299] Negotiation successful
0929 19:13:03.243854 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 19:13:03.244385 25798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22b7c3180cca4bfc9f0150237e65c76e" candidate_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8375ab687994ea6868fd4b1d054b00e" is_pre_election: true
I20260929 19:13:03.244387 25664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22b7c3180cca4bfc9f0150237e65c76e" candidate_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" is_pre_election: true
I20260929 19:13:03.244546 25798 raft_consensus.cc:2454] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0be3da1d3f24a4ab5a5c5dd9ac1b53d in term 0.
I20260929 19:13:03.244546 25664 raft_consensus.cc:2454] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0be3da1d3f24a4ab5a5c5dd9ac1b53d in term 0.
I20260929 19:13:03.244781 25867 leader_election.cc:302] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0be3da1d3f24a4ab5a5c5dd9ac1b53d, f8375ab687994ea6868fd4b1d054b00e; no voters:
I20260929 19:13:03.244961 26002 raft_consensus.cc:2781] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:13:03.245050 26002 raft_consensus.cc:482] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260929 19:13:03.245038 25711 tablet.cc:2369] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:13:03.245079 26002 raft_consensus.cc:3037] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:13:03.246023 26002 raft_consensus.cc:504] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:03.246274 26002 leader_election.cc:288] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [CANDIDATE]: Term 1 election: Requested vote from peers f8375ab687994ea6868fd4b1d054b00e (127.9.236.2:42955), fc1d458bd6ff4dd39b4cb0303b7c4dbb (127.9.236.1:41083)
I20260929 19:13:03.246364 25798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22b7c3180cca4bfc9f0150237e65c76e" candidate_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8375ab687994ea6868fd4b1d054b00e"
I20260929 19:13:03.246402 25664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22b7c3180cca4bfc9f0150237e65c76e" candidate_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb"
I20260929 19:13:03.246464 25664 raft_consensus.cc:3037] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:13:03.246462 25798 raft_consensus.cc:3037] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:13:03.247433 25798 raft_consensus.cc:2454] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0be3da1d3f24a4ab5a5c5dd9ac1b53d in term 1.
I20260929 19:13:03.247433 25664 raft_consensus.cc:2454] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0be3da1d3f24a4ab5a5c5dd9ac1b53d in term 1.
I20260929 19:13:03.247650 25867 leader_election.cc:302] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0be3da1d3f24a4ab5a5c5dd9ac1b53d, f8375ab687994ea6868fd4b1d054b00e; no voters:
I20260929 19:13:03.247756 26002 raft_consensus.cc:2781] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:13:03.247936 26002 raft_consensus.cc:686] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 1 LEADER]: Becoming Leader. State: Replica: d0be3da1d3f24a4ab5a5c5dd9ac1b53d, State: Running, Role: LEADER
I20260929 19:13:03.248052 26002 consensus_queue.cc:237] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:03.248993 25518 catalog_manager.cc:5495] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d reported cstate change: term changed from 0 to 1, leader changed from <none> to d0be3da1d3f24a4ab5a5c5dd9ac1b53d (127.9.236.3). New cstate: current_term: 1 leader_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } health_report { overall_health: UNKNOWN } } }
I20260929 19:13:03.285845 25994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:03.283295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56060 (local address 127.9.236.3:35887)
0929 19:13:03.283402 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:03.283404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:03.283414 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:13:03.283467 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:03.283468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:03.283495 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:03.283563 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:13:03.283736 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.284261 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.285038 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:03.285282 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:03.285336 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:03.285472 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:13:03.285599 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:13:03.285677 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:03.285697 (+ 20us) server_negotiation.cc:299] Negotiation successful
0929 19:13:03.285722 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 19:13:03.287904 25798 raft_consensus.cc:1261] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e [term 1 FOLLOWER]: Refusing update from remote peer d0be3da1d3f24a4ab5a5c5dd9ac1b53d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:13:03.287979 25664 raft_consensus.cc:1261] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Refusing update from remote peer d0be3da1d3f24a4ab5a5c5dd9ac1b53d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:13:03.288262 26005 consensus_queue.cc:1035] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:13:03.288409 26002 consensus_queue.cc:1035] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:13:03.291395 26009 mvcc.cc:205] Tried to move back new op lower bound from 7334744814744301568 to 7334744814584725504. Current Snapshot: MvccSnapshot[applied={T|T < 7334744814744301568}]
I20260929 19:13:03.291524 26008 mvcc.cc:205] Tried to move back new op lower bound from 7334744814744301568 to 7334744814584725504. Current Snapshot: MvccSnapshot[applied={T|T < 7334744814744301568}]
I20260929 19:13:03.293617 25517 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:13:03.293702 25517 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:13:03.294423 25517 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069: Sending DeleteTablet for 3 replicas of tablet 22b7c3180cca4bfc9f0150237e65c76e
I20260929 19:13:03.294729 25912 tablet_service.cc:1514] Processing DeleteTablet for tablet 22b7c3180cca4bfc9f0150237e65c76e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:13:03 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:56046
I20260929 19:13:03.294735 25778 tablet_service.cc:1514] Processing DeleteTablet for tablet 22b7c3180cca4bfc9f0150237e65c76e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:13:03 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:54900
I20260929 19:13:03.294775 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 25581
I20260929 19:13:03.295009 26018 tablet_replica.cc:330] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e: stopping tablet replica
I20260929 19:13:03.295094 26018 raft_consensus.cc:2229] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:13:03.295187 26018 raft_consensus.cc:2258] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:13:03.297283 26019 tablet_replica.cc:330] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: stopping tablet replica
I20260929 19:13:03.297384 26019 raft_consensus.cc:2229] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:13:03.297515 26019 raft_consensus.cc:2258] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:13:03.297932 26019 ts_tablet_manager.cc:1905] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:13:03.299690 26019 ts_tablet_manager.cc:1918] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:13:03.299769 26019 log.cc:1196] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/22b7c3180cca4bfc9f0150237e65c76e
I20260929 19:13:03.299998 26019 ts_tablet_manager.cc:1939] T 22b7c3180cca4bfc9f0150237e65c76e P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: Deleting consensus metadata
I20260929 19:13:03.300352 25506 catalog_manager.cc:4841] TS d0be3da1d3f24a4ab5a5c5dd9ac1b53d (127.9.236.3:35887): tablet 22b7c3180cca4bfc9f0150237e65c76e (table test-table [id=411fc4e732a546e7af6d06a50846e05f]) successfully deleted
I20260929 19:13:03.301455 26018 ts_tablet_manager.cc:1905] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 19:13:03.301458 25503 connection.cc:535] client connection to 127.9.236.1:41083 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 19:13:03.301553 25503 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 19:13:03.302094 25991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:03.301964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:41083 (local address 127.0.0.1:53740)
0929 19:13:03.302019 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:03.302055 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.1:41083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 19:13:03.302213 25503 catalog_manager.cc:4584] TS fc1d458bd6ff4dd39b4cb0303b7c4dbb (127.9.236.1:41083): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 22b7c3180cca4bfc9f0150237e65c76e: Network error: Client connection negotiation failed: client connection to 127.9.236.1:41083: connect: Connection refused (error 111)
I20260929 19:13:03.302621 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 25715
I20260929 19:13:03.307768 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 25849
I20260929 19:13:03.307933 25991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:03.307780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:42955 (local address 127.0.0.1:54914)
0929 19:13:03.307859 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:03.307896 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.2:42955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260929 19:13:03.312930 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 25487
I20260929 19:13:23.318295 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:37539
--webserver_interface=127.9.236.62
--webserver_port=40689
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:37539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:13:23.408865 26021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:13:23.409130 26021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:13:23.409166 26021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:13:23.410440 26021 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 19:13:23.410485 26021 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 19:13:23.410496 26021 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 19:13:23.410506 26021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:13:23.410522 26021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:13:23.410550 26021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:13:23.410566 26021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:13:23.410588 26021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:13:23.412141 26021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:37539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:37539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=40689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:13:23.412403 26021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:13:23.412640 26021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:13:23.422189 26021 server_base.cc:1034] running on GCE node
W20260929 19:13:23.422318 26028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:13:23.422334 26027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:13:23.422380 26030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:13:23.422801 26021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:13:23.423151 26021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:13:23.424309 26021 hybrid_clock.cc:648] HybridClock initialized: now 1790709203424296 us; error 44 us; skew 500 ppm
Sep 29 19:13:23 dist-test-slave-wftq krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709203, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:13:23.427084 26021 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:13:23.427537 26021 webserver.cc:460] Webserver started at http://127.9.236.62:40689/ using document root <none> and password file <none>
I20260929 19:13:23.427750 26021 fs_manager.cc:359] Metadata directory not provided
I20260929 19:13:23.427802 26021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:13:23.429273 26021 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:13:23.429975 26037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:23.430166 26021 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:13:23.430259 26021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "b9850d4c2a4043cd86865de232d53069"
format_stamp: "Formatted at 2026-09-29 19:13:02 on dist-test-slave-wftq"
server_key: "acd5f8dacf21040a545707bdda8da682"
server_key_iv: "6a43419585487c1426b34cf37f36b63d"
server_key_version: "encryptionkey@0"
I20260929 19:13:23.430709 26021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:13:23.449606 26021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:13:23.451577 26021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:13:23.451711 26021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:13:23.455941 26021 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:37539
I20260929 19:13:23.456013 26089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:37539 every 8 connection(s)
I20260929 19:13:23.456386 26021 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:13:23.458209 26090 sys_catalog.cc:263] Verifying existing consensus state
I20260929 19:13:23.458989 26090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069: Bootstrap starting.
I20260929 19:13:23.464138 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 26021
I20260929 19:13:23.464548 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:41083
--local_ip_for_outbound_sockets=127.9.236.1
--tserver_master_addrs=127.9.236.62:37539
--webserver_port=43677
--webserver_interface=127.9.236.1
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:13:23.469779 26090 log.cc:824] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069: Log is configured to *not* fsync() on all Append() calls
I20260929 19:13:23.471959 26090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:13:23.472304 26090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069: Bootstrap complete.
I20260929 19:13:23.474687 26090 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9850d4c2a4043cd86865de232d53069" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37539 } }
I20260929 19:13:23.474948 26090 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9850d4c2a4043cd86865de232d53069, State: Initialized, Role: FOLLOWER
I20260929 19:13:23.475119 26090 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9850d4c2a4043cd86865de232d53069" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37539 } }
I20260929 19:13:23.475198 26090 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:13:23.475236 26090 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:13:23.475291 26090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 1 FOLLOWER]: Advancing to term 2
I20260929 19:13:23.476460 26090 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9850d4c2a4043cd86865de232d53069" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37539 } }
I20260929 19:13:23.476580 26090 leader_election.cc:302] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9850d4c2a4043cd86865de232d53069; no voters:
I20260929 19:13:23.476785 26090 leader_election.cc:288] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 19:13:23.476836 26094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 19:13:23.477082 26090 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:13:23.477077 26094 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 2 LEADER]: Becoming Leader. State: Replica: b9850d4c2a4043cd86865de232d53069, State: Running, Role: LEADER
I20260929 19:13:23.477401 26094 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9850d4c2a4043cd86865de232d53069" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37539 } }
I20260929 19:13:23.477867 26095 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b9850d4c2a4043cd86865de232d53069" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9850d4c2a4043cd86865de232d53069" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37539 } } }
I20260929 19:13:23.477952 26095 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [sys.catalog]: This master's current role is: LEADER
I20260929 19:13:23.478199 26103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:13:23.478264 26096 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9850d4c2a4043cd86865de232d53069. Latest consensus state: current_term: 2 leader_uuid: "b9850d4c2a4043cd86865de232d53069" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9850d4c2a4043cd86865de232d53069" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37539 } } }
I20260929 19:13:23.478386 26096 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [sys.catalog]: This master's current role is: LEADER
I20260929 19:13:23.479290 26103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:13:23.479456 26103 catalog_manager.cc:1259] Loaded cluster ID: 3a30a93c8b4045a6a4616894898973c9
I20260929 19:13:23.479681 26103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:13:23.480844 26103 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:13:23.481052 26103 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069: Loaded TSK: 0
I20260929 19:13:23.491318 26103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069: Generated new TSK 1
I20260929 19:13:23.491427 26103 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 19:13:23.550199 26092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:13:23.550390 26092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:13:23.550420 26092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:13:23.551535 26092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:13:23.551576 26092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:13:23.551609 26092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:13:23.553004 26092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:41083
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=43677
--tserver_master_addrs=127.9.236.62:37539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:13:23.553253 26092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:13:23.553457 26092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:13:23.559783 26092 server_base.cc:1034] running on GCE node
W20260929 19:13:23.559746 26118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:13:23.559973 26117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:13:23.559764 26120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:13:23.560411 26092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:13:23.560647 26092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:13:23.561785 26092 hybrid_clock.cc:648] HybridClock initialized: now 1790709203561767 us; error 47 us; skew 500 ppm
Sep 29 19:13:23 dist-test-slave-wftq krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709203, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:13:23.564779 26092 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:13:23.565480 26092 webserver.cc:460] Webserver started at http://127.9.236.1:43677/ using document root <none> and password file <none>
I20260929 19:13:23.565794 26092 fs_manager.cc:359] Metadata directory not provided
I20260929 19:13:23.565886 26092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:13:23.567801 26092 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:13:23.568660 26127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:23.568871 26092 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:13:23.568966 26092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb"
format_stamp: "Formatted at 2026-09-29 19:13:02 on dist-test-slave-wftq"
server_key: "16eedb70479a5c26aa6127febe4b36dc"
server_key_iv: "09c088a63334f424604990ff1da0f2d1"
server_key_version: "encryptionkey@0"
I20260929 19:13:23.569545 26092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:13:23.585997 26092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:13:23.588536 26092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:13:23.588697 26092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:13:23.588992 26092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:13:23.589596 26134 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 19:13:23.591199 26092 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 19:13:23.591265 26092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260929 19:13:23.591303 26092 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 19:13:23.592358 26092 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 19:13:23.592409 26092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260929 19:13:23.592486 26134 tablet_bootstrap.cc:492] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Bootstrap starting.
I20260929 19:13:23.600927 26092 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:41083
I20260929 19:13:23.601266 26241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:41083 every 8 connection(s)
I20260929 19:13:23.601529 26092 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:13:23.602774 26134 log.cc:824] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Log is configured to *not* fsync() on all Append() calls
I20260929 19:13:23.604882 26134 tablet_bootstrap.cc:492] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:13:23.605188 26134 tablet_bootstrap.cc:492] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Bootstrap complete.
I20260929 19:13:23.606009 26134 ts_tablet_manager.cc:1397] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.006s
I20260929 19:13:23.607331 26134 raft_consensus.cc:348] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:23.607616 26134 raft_consensus.cc:729] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1d458bd6ff4dd39b4cb0303b7c4dbb, State: Initialized, Role: FOLLOWER
I20260929 19:13:23.607785 26134 consensus_queue.cc:260] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:23.608014 26134 ts_tablet_manager.cc:1428] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:13:23.610631 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 26092
I20260929 19:13:23.610961 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:42955
--local_ip_for_outbound_sockets=127.9.236.2
--tserver_master_addrs=127.9.236.62:37539
--webserver_port=34799
--webserver_interface=127.9.236.2
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 19:13:23 dist-test-slave-wftq krb5kdc[25474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709203, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:13:23.617530 26244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:23.602459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.1:51387)
0929 19:13:23.605226 (+ 2767us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:23.605276 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 19:13:23.606007 (+ 731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:13:23.606575 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:13:23.606581 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:13:23.606961 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:13:23.607240 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:23.607247 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.609051 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.609052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:23.609991 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:23.609994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.610872 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.610873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:23.611018 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:23.611775 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:13:23.611784 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:13:23.613272 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:13:23.615357 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:13:23.615369 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:13:23.615372 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:13:23.615795 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:13:23.616144 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:13:23.616145 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:13:23.616145 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:13:23.616288 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:13:23.616779 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:13:23.616786 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:13:23.616953 (+ 167us) client_negotiation.cc:769] Sending connection context
0929 19:13:23.617066 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 19:13:23.617303 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2714,"thread_start_us":106,"threads_started":1}
I20260929 19:13:23.617564 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:23.602579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:51387 (local address 127.9.236.62:37539)
0929 19:13:23.602818 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:23.602828 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:23.605441 (+ 2613us) server_negotiation.cc:407] Connection header received
0929 19:13:23.606303 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:23.606310 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:23.606366 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:23.606492 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:13:23.607392 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.608914 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.610095 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.610746 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.610866 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:23.613403 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:13:23.613407 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:13:23.613408 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:13:23.613421 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:13:23.615241 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:23.615917 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:23.615920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:23.615921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:23.616054 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:23.616377 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:23.616378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:23.616378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:23.616693 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:13:23.616785 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:23.617188 (+ 403us) server_negotiation.cc:299] Negotiation successful
0929 19:13:23.617352 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":121,"threads_started":1}
I20260929 19:13:23.618505 26242 heartbeater.cc:347] Connected to a master server at 127.9.236.62:37539
I20260929 19:13:23.618613 26242 heartbeater.cc:464] Registering TS with master...
I20260929 19:13:23.618832 26242 heartbeater.cc:511] Master 127.9.236.62:37539 requested a full tablet report, sending...
I20260929 19:13:23.619630 26054 ts_manager.cc:194] Registered new tserver with Master: fc1d458bd6ff4dd39b4cb0303b7c4dbb (127.9.236.1:41083)
I20260929 19:13:23.620869 26054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:51387
I20260929 19:13:23.621953 26242 heartbeater.cc:503] Master 127.9.236.62:37539 was elected leader, sending a full tablet report...
Sep 29 19:13:23 dist-test-slave-wftq krb5kdc[25474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709203, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for kudu/127.9.236.1@KRBTEST.COM
I20260929 19:13:23.629694 26250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:23.620602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54578 (local address 127.9.236.1:41083)
0929 19:13:23.620776 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:23.620780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:23.621319 (+ 539us) server_negotiation.cc:407] Connection header received
0929 19:13:23.621475 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:23.621479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:23.621499 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:23.621553 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:13:23.622255 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.623226 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.624228 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.624603 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.624644 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:23.627040 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:13:23.627047 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:13:23.627047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:13:23.627054 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:13:23.628497 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:23.628853 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:23.628855 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:23.628856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:23.628910 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:23.629074 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:23.629075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:23.629075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:23.629261 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:13:23.629337 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:23.629502 (+ 165us) server_negotiation.cc:299] Negotiation successful
0929 19:13:23.629597 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
I20260929 19:13:23.629694 26249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:23.620322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:41083 (local address 127.0.0.1:54578)
0929 19:13:23.621214 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:23.621253 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 19:13:23.621365 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:13:23.621653 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:13:23.621657 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:13:23.622060 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:13:23.622155 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:23.622158 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.623322 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.623323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:23.624140 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:23.624142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.624722 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.624723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:23.624788 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:23.625299 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:13:23.625304 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:13:23.626952 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:13:23.628576 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:13:23.628582 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:13:23.628584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:13:23.628782 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:13:23.628966 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:13:23.628967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:13:23.628968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:13:23.629026 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:13:23.629325 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:13:23.629331 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:13:23.629386 (+ 55us) client_negotiation.cc:769] Sending connection context
0929 19:13:23.629461 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 19:13:23.629586 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"thread_start_us":110,"threads_started":1}
I20260929 19:13:23.630388 26174 tablet_service.cc:1514] Processing DeleteTablet for tablet 22b7c3180cca4bfc9f0150237e65c76e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:13:03 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:54578
I20260929 19:13:23.631047 26251 tablet_replica.cc:330] stopping tablet replica
I20260929 19:13:23.631158 26251 raft_consensus.cc:2229] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:13:23.631263 26251 raft_consensus.cc:2258] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:13:23.631600 26251 ts_tablet_manager.cc:1905] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:13:23.633903 26251 ts_tablet_manager.cc:1918] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:13:23.633966 26251 log.cc:1196] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/wal/wals/22b7c3180cca4bfc9f0150237e65c76e
I20260929 19:13:23.634181 26251 ts_tablet_manager.cc:1939] T 22b7c3180cca4bfc9f0150237e65c76e P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Deleting consensus metadata
I20260929 19:13:23.634732 26038 catalog_manager.cc:4841] TS fc1d458bd6ff4dd39b4cb0303b7c4dbb (127.9.236.1:41083): tablet 22b7c3180cca4bfc9f0150237e65c76e (table test-table [id=411fc4e732a546e7af6d06a50846e05f]) successfully deleted
W20260929 19:13:23.694900 26248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:13:23.695115 26248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:13:23.695138 26248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:13:23.696224 26248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:13:23.696265 26248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:13:23.696290 26248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:13:23.697697 26248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:42955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=34799
--tserver_master_addrs=127.9.236.62:37539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:13:23.697880 26248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:13:23.698067 26248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:13:23.703147 26257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:13:23.703173 26259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:13:23.703224 26256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:13:23.703397 26248 server_base.cc:1034] running on GCE node
I20260929 19:13:23.703596 26248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:13:23.703794 26248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:13:23.704931 26248 hybrid_clock.cc:648] HybridClock initialized: now 1790709203704915 us; error 36 us; skew 500 ppm
Sep 29 19:13:23 dist-test-slave-wftq krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709203, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:13:23.707516 26248 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:13:23.707993 26248 webserver.cc:460] Webserver started at http://127.9.236.2:34799/ using document root <none> and password file <none>
I20260929 19:13:23.708175 26248 fs_manager.cc:359] Metadata directory not provided
I20260929 19:13:23.708227 26248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:13:23.709589 26248 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:13:23.710279 26266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:23.710474 26248 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 19:13:23.710573 26248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "f8375ab687994ea6868fd4b1d054b00e"
format_stamp: "Formatted at 2026-09-29 19:13:02 on dist-test-slave-wftq"
server_key: "ce9fc6758ea6e61a1be921381b9ee3de"
server_key_iv: "6ef94e67ffc09c1d95234a67f774743d"
server_key_version: "encryptionkey@0"
I20260929 19:13:23.711012 26248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:13:23.727819 26248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:13:23.729908 26248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:13:23.730026 26248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:13:23.730240 26248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:13:23.730674 26273 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260929 19:13:23.732103 26273 ts_tablet_manager.cc:1852] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260929 19:13:23.732395 26273 ts_tablet_manager.cc:1905] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:13:23.734549 26273 ts_tablet_manager.cc:1918] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:13:23.734620 26273 log.cc:1196] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/22b7c3180cca4bfc9f0150237e65c76e
I20260929 19:13:23.734844 26273 ts_tablet_manager.cc:1939] T 22b7c3180cca4bfc9f0150237e65c76e P f8375ab687994ea6868fd4b1d054b00e: Deleting consensus metadata
I20260929 19:13:23.735034 26248 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260929 19:13:23.735085 26248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260929 19:13:23.735110 26248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:13:23.735121 26248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:23.742014 26248 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:42955
I20260929 19:13:23.742120 26380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:42955 every 8 connection(s)
I20260929 19:13:23.742352 26248 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:13:23.747089 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 26248
I20260929 19:13:23.747334 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:35887
--local_ip_for_outbound_sockets=127.9.236.3
--tserver_master_addrs=127.9.236.62:37539
--webserver_port=42627
--webserver_interface=127.9.236.3
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 19:13:23 dist-test-slave-wftq krb5kdc[25474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709203, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:13:23.752177 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:23.743112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:45837 (local address 127.9.236.62:37539)
0929 19:13:23.743193 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:23.743196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:23.743405 (+ 209us) server_negotiation.cc:407] Connection header received
0929 19:13:23.744038 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:23.744040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:23.744063 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:23.744118 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:13:23.744788 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.745383 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.746411 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.746621 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.746683 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:23.749239 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:13:23.749241 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:13:23.749242 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:13:23.749250 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:13:23.750647 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:23.751067 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:23.751068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:23.751069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:23.751128 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:23.751446 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:23.751448 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:23.751449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:23.751596 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:13:23.751660 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:23.752006 (+ 346us) server_negotiation.cc:299] Negotiation successful
0929 19:13:23.752107 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 19:13:23.752420 26383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:23.743058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.2:45837)
0929 19:13:23.743266 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:23.743307 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:13:23.743931 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:13:23.744208 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:13:23.744212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:13:23.744481 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:13:23.744701 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:23.744705 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.745472 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.745473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:23.746335 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:23.746337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.746695 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.746696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:23.746824 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:23.747374 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:13:23.747383 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:13:23.749093 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:13:23.750750 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:13:23.750756 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:13:23.750758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:13:23.750993 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:13:23.751213 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:13:23.751214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:13:23.751215 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:13:23.751359 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:13:23.751673 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:13:23.751679 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:13:23.751861 (+ 182us) client_negotiation.cc:769] Sending connection context
0929 19:13:23.751987 (+ 126us) client_negotiation.cc:240] Negotiation successful
0929 19:13:23.752168 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":100,"threads_started":1}
I20260929 19:13:23.753019 26381 heartbeater.cc:347] Connected to a master server at 127.9.236.62:37539
I20260929 19:13:23.753147 26381 heartbeater.cc:464] Registering TS with master...
I20260929 19:13:23.753334 26381 heartbeater.cc:511] Master 127.9.236.62:37539 requested a full tablet report, sending...
I20260929 19:13:23.753743 26054 ts_manager.cc:194] Registered new tserver with Master: f8375ab687994ea6868fd4b1d054b00e (127.9.236.2:42955)
I20260929 19:13:23.754106 26054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:45837
W20260929 19:13:23.834398 26384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:13:23.834633 26384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:13:23.834658 26384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:13:23.835840 26384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:13:23.835880 26384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:13:23.835906 26384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:13:23.837396 26384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:35887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=42627
--tserver_master_addrs=127.9.236.62:37539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:13:23.837625 26384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:13:23.837845 26384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:13:23.843101 26389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:13:23.843281 26390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:13:23.843293 26384 server_base.cc:1034] running on GCE node
W20260929 19:13:23.843101 26392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:13:23.843662 26384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:13:23.843964 26384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:13:23.845144 26384 hybrid_clock.cc:648] HybridClock initialized: now 1790709203845124 us; error 39 us; skew 500 ppm
Sep 29 19:13:23 dist-test-slave-wftq krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709203, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:13:23.847592 26384 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:13:23.848033 26384 webserver.cc:460] Webserver started at http://127.9.236.3:42627/ using document root <none> and password file <none>
I20260929 19:13:23.848220 26384 fs_manager.cc:359] Metadata directory not provided
I20260929 19:13:23.848271 26384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:13:23.849705 26384 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:13:23.850284 26399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:23.850499 26384 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:13:23.850582 26384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d"
format_stamp: "Formatted at 2026-09-29 19:13:03 on dist-test-slave-wftq"
server_key: "0322d575d2c7ea48b37db87f083be7d1"
server_key_iv: "59a8347e8ad657f3f6378aa4fd70b23f"
server_key_version: "encryptionkey@0"
I20260929 19:13:23.850968 26384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:13:23.861393 26384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:13:23.863363 26384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:13:23.863485 26384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:13:23.863710 26384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:13:23.864046 26384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:13:23.864084 26384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:23.864115 26384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:13:23.864132 26384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:23.870640 26384 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:35887
I20260929 19:13:23.870774 26512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:35887 every 8 connection(s)
I20260929 19:13:23.871052 26384 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:13:23.873029 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 26384
Sep 29 19:13:23 dist-test-slave-wftq krb5kdc[25474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709203, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:13:23.881964 26517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:23.873651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51768 (local address 127.9.236.62:37539)
0929 19:13:23.873788 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:23.873790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:23.873850 (+ 60us) server_negotiation.cc:407] Connection header received
0929 19:13:23.873913 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:23.873913 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:23.873929 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:23.874029 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:13:23.874789 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.875505 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.876503 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.876902 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.876986 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:23.878553 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:13:23.878555 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:13:23.878556 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:13:23.878564 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:13:23.880774 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:23.881219 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:23.881220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:23.881220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:23.881272 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:23.881481 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:23.881482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:23.881482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:23.881598 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:13:23.881768 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:23.881802 (+ 34us) server_negotiation.cc:299] Negotiation successful
0929 19:13:23.881873 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":72,"threads_started":1}
I20260929 19:13:23.882871 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:23.871895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:39183 (local address 127.9.236.62:37539)
0929 19:13:23.871968 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:23.871971 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:23.872215 (+ 244us) server_negotiation.cc:407] Connection header received
0929 19:13:23.872900 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:23.872901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:23.872914 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:23.873008 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:13:23.874403 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.875063 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.876325 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.876670 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.876730 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:23.879538 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:13:23.879539 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:13:23.879539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:13:23.879547 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:13:23.881147 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:23.881706 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:23.881708 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:23.881708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:23.881768 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:23.882160 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:23.882161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:23.882162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:23.882313 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:13:23.882383 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:23.882733 (+ 350us) server_negotiation.cc:299] Negotiation successful
0929 19:13:23.882815 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 19:13:23.883123 26515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:23.871845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.3:39183)
0929 19:13:23.872060 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:23.872109 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 19:13:23.872748 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:13:23.873144 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:13:23.873150 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:13:23.873988 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:13:23.874290 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:23.874298 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.875187 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.875188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:23.876233 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:23.876237 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.876795 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.876796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:23.876959 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:23.877620 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:13:23.877679 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:13:23.879436 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:13:23.881250 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:13:23.881262 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:13:23.881265 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:13:23.881619 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:13:23.881835 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:13:23.881836 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:13:23.881836 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:13:23.882026 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:13:23.882396 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:13:23.882403 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:13:23.882577 (+ 174us) client_negotiation.cc:769] Sending connection context
0929 19:13:23.882698 (+ 121us) client_negotiation.cc:240] Negotiation successful
0929 19:13:23.882886 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":117,"threads_started":1}
I20260929 19:13:23.883643 26513 heartbeater.cc:347] Connected to a master server at 127.9.236.62:37539
I20260929 19:13:23.883770 26513 heartbeater.cc:464] Registering TS with master...
I20260929 19:13:23.884001 26513 heartbeater.cc:511] Master 127.9.236.62:37539 requested a full tablet report, sending...
I20260929 19:13:23.884423 26054 ts_manager.cc:194] Registered new tserver with Master: d0be3da1d3f24a4ab5a5c5dd9ac1b53d (127.9.236.3:35887)
I20260929 19:13:23.884814 26054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:39183
I20260929 19:13:23.893949 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:13:23.896903 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:23.894396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51776 (local address 127.9.236.62:37539)
0929 19:13:23.894465 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:23.894467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:23.894514 (+ 47us) server_negotiation.cc:407] Connection header received
0929 19:13:23.894610 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:23.894611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:23.894675 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:23.894740 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:13:23.894904 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.895424 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.896213 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.896402 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.896457 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:23.896600 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:13:23.896714 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:13:23.896792 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:23.896809 (+ 17us) server_negotiation.cc:299] Negotiation successful
0929 19:13:23.896829 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 19:13:23.897279 26054 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:51776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:13:23.899209 26054 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:13:23.901758 26174 tablet_service.cc:1467] Processing CreateTablet for tablet 9edb3e5acbee4165a87480cf3b33fbdb (DEFAULT_TABLE table=test-table [id=abcbf14f7b044a51ba7993bcfad61c9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:13:23.902004 26174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9edb3e5acbee4165a87480cf3b33fbdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:13:23.903288 26134 tablet_bootstrap.cc:492] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Bootstrap starting.
I20260929 19:13:23.903620 26134 tablet_bootstrap.cc:654] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Neither blocks nor log segments found. Creating new log.
I20260929 19:13:23.904145 26134 tablet_bootstrap.cc:492] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb: No bootstrap required, opened a new log
I20260929 19:13:23.904188 26134 ts_tablet_manager.cc:1397] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260929 19:13:23.904294 26134 raft_consensus.cc:348] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:23.904366 26134 raft_consensus.cc:374] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:13:23.904410 26134 raft_consensus.cc:729] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1d458bd6ff4dd39b4cb0303b7c4dbb, State: Initialized, Role: FOLLOWER
I20260929 19:13:23.904454 26134 consensus_queue.cc:260] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:23.904541 26134 ts_tablet_manager.cc:1428] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:23.905980 26520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:23.901618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33926 (local address 127.9.236.3:35887)
0929 19:13:23.901990 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:23.901994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:23.902003 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:13:23.902038 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:23.902040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:23.902054 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:23.902146 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:13:23.902307 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.903406 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.904860 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.905555 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.905627 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:23.905697 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:23.905842 (+ 145us) server_negotiation.cc:299] Negotiation successful
0929 19:13:23.905887 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":93,"threads_started":1}
I20260929 19:13:23.906018 26521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:23.901783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60028 (local address 127.9.236.2:42955)
0929 19:13:23.901958 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:23.901964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:23.901978 (+ 14us) server_negotiation.cc:407] Connection header received
0929 19:13:23.902023 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:23.902025 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:23.902043 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:23.902205 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:13:23.902437 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.903494 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.904849 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.905670 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.905743 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:23.905804 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:23.905896 (+ 92us) server_negotiation.cc:299] Negotiation successful
0929 19:13:23.905937 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":88,"threads_started":1}
I20260929 19:13:23.905993 26249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:23.901407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:35887 (local address 127.0.0.1:33926)
0929 19:13:23.901463 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:23.901473 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:13:23.901513 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:13:23.902168 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:13:23.902169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:13:23.902171 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:13:23.902231 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:23.902233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.903510 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.903511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:23.904758 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:23.904760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.905645 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.905646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:23.905736 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:23.905761 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 19:13:23.905819 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 19:13:23.905896 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":4}
I20260929 19:13:23.905998 26519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:23.901470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:42955 (local address 127.0.0.1:60028)
0929 19:13:23.901668 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:23.901678 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:13:23.901718 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:13:23.902326 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:13:23.902327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:13:23.902328 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:13:23.902367 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:23.902367 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.903577 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.903578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:23.904762 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:23.904764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:23.905746 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:23.905747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:23.905806 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:23.905812 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 19:13:23.905865 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 19:13:23.905914 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":9,"thread_start_us":135,"threads_started":1}
I20260929 19:13:23.906764 26447 tablet_service.cc:1467] Processing CreateTablet for tablet 9edb3e5acbee4165a87480cf3b33fbdb (DEFAULT_TABLE table=test-table [id=abcbf14f7b044a51ba7993bcfad61c9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:13:23.907068 26447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9edb3e5acbee4165a87480cf3b33fbdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:13:23.906988 26315 tablet_service.cc:1467] Processing CreateTablet for tablet 9edb3e5acbee4165a87480cf3b33fbdb (DEFAULT_TABLE table=test-table [id=abcbf14f7b044a51ba7993bcfad61c9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:13:23.907335 26315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9edb3e5acbee4165a87480cf3b33fbdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:13:23.909346 26273 tablet_bootstrap.cc:492] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e: Bootstrap starting.
I20260929 19:13:23.909932 26523 tablet_bootstrap.cc:492] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: Bootstrap starting.
I20260929 19:13:23.910216 26273 tablet_bootstrap.cc:654] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e: Neither blocks nor log segments found. Creating new log.
I20260929 19:13:23.910601 26273 log.cc:824] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e: Log is configured to *not* fsync() on all Append() calls
I20260929 19:13:23.910697 26523 tablet_bootstrap.cc:654] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: Neither blocks nor log segments found. Creating new log.
I20260929 19:13:23.911263 26523 log.cc:824] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: Log is configured to *not* fsync() on all Append() calls
I20260929 19:13:23.911440 26273 tablet_bootstrap.cc:492] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e: No bootstrap required, opened a new log
I20260929 19:13:23.911520 26273 ts_tablet_manager.cc:1397] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:13:23.912016 26523 tablet_bootstrap.cc:492] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: No bootstrap required, opened a new log
I20260929 19:13:23.912082 26523 ts_tablet_manager.cc:1397] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:13:23.913004 26273 raft_consensus.cc:348] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:23.913189 26273 raft_consensus.cc:374] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:13:23.913216 26273 raft_consensus.cc:729] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8375ab687994ea6868fd4b1d054b00e, State: Initialized, Role: FOLLOWER
I20260929 19:13:23.913316 26273 consensus_queue.cc:260] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:23.913592 26273 ts_tablet_manager.cc:1428] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:13:23.913650 26381 heartbeater.cc:503] Master 127.9.236.62:37539 was elected leader, sending a full tablet report...
I20260929 19:13:23.913822 26523 raft_consensus.cc:348] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:23.913995 26523 raft_consensus.cc:374] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:13:23.914034 26523 raft_consensus.cc:729] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0be3da1d3f24a4ab5a5c5dd9ac1b53d, State: Initialized, Role: FOLLOWER
I20260929 19:13:23.914151 26523 consensus_queue.cc:260] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:23.914420 26523 ts_tablet_manager.cc:1428] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 19:13:23.914556 26513 heartbeater.cc:503] Master 127.9.236.62:37539 was elected leader, sending a full tablet report...
W20260929 19:13:23.992944 26382 tablet.cc:2369] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:13:24.100446 26526 raft_consensus.cc:482] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:13:24.100580 26526 raft_consensus.cc:504] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:24.101135 26526 leader_election.cc:288] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0be3da1d3f24a4ab5a5c5dd9ac1b53d (127.9.236.3:35887), fc1d458bd6ff4dd39b4cb0303b7c4dbb (127.9.236.1:41083)
I20260929 19:13:24.104285 26383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:24.101326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:35887 (local address 127.9.236.2:35053)
0929 19:13:24.101428 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:24.101441 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:13:24.101498 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:13:24.101635 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:13:24.101636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:13:24.101638 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:13:24.101720 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:24.101723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:24.102512 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:24.102513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:24.103333 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:24.103335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:24.104011 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:24.104013 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:24.104099 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:24.104104 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:13:24.104169 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 19:13:24.104212 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260929 19:13:24.104281 26520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:24.101445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:35053 (local address 127.9.236.3:35887)
0929 19:13:24.101513 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:24.101515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:24.101525 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:13:24.101557 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:24.101557 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:24.101570 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:24.101622 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:13:24.101826 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:24.102425 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:24.103437 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:24.103929 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:24.103986 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:24.103998 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:24.104178 (+ 180us) server_negotiation.cc:299] Negotiation successful
0929 19:13:24.104211 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 19:13:24.104732 26250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:24.101519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:41263 (local address 127.9.236.1:41083)
0929 19:13:24.101595 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:24.101598 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:24.101703 (+ 105us) server_negotiation.cc:407] Connection header received
0929 19:13:24.101769 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:24.101770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:24.101798 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:24.101871 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:13:24.102072 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:24.102635 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:24.103738 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:24.104269 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:24.104333 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:24.104406 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:24.104628 (+ 222us) server_negotiation.cc:299] Negotiation successful
0929 19:13:24.104669 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 19:13:24.104739 26528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:24.101414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:41083 (local address 127.9.236.2:41263)
0929 19:13:24.101616 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:24.101629 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:13:24.101708 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:13:24.101894 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:13:24.101895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:13:24.101897 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:13:24.101990 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:24.101993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:24.102731 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:24.102732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:24.103641 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:24.103644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:24.104477 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:24.104478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:24.104542 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:24.104547 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:13:24.104609 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 19:13:24.104660 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":138,"threads_started":1}
I20260929 19:13:24.104759 26467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9edb3e5acbee4165a87480cf3b33fbdb" candidate_uuid: "f8375ab687994ea6868fd4b1d054b00e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" is_pre_election: true
I20260929 19:13:24.104948 26467 raft_consensus.cc:2454] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8375ab687994ea6868fd4b1d054b00e in term 0.
I20260929 19:13:24.105093 26196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9edb3e5acbee4165a87480cf3b33fbdb" candidate_uuid: "f8375ab687994ea6868fd4b1d054b00e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" is_pre_election: true
I20260929 19:13:24.105283 26196 raft_consensus.cc:2454] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8375ab687994ea6868fd4b1d054b00e in term 0.
I20260929 19:13:24.105263 26270 leader_election.cc:302] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0be3da1d3f24a4ab5a5c5dd9ac1b53d, f8375ab687994ea6868fd4b1d054b00e; no voters:
I20260929 19:13:24.105540 26526 raft_consensus.cc:2781] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:13:24.105649 26526 raft_consensus.cc:482] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:13:24.105674 26526 raft_consensus.cc:3037] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e [term 0 FOLLOWER]: Advancing to term 1
W20260929 19:13:24.105796 26243 tablet.cc:2369] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:13:24.106597 26526 raft_consensus.cc:504] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:24.106748 26526 leader_election.cc:288] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e [CANDIDATE]: Term 1 election: Requested vote from peers d0be3da1d3f24a4ab5a5c5dd9ac1b53d (127.9.236.3:35887), fc1d458bd6ff4dd39b4cb0303b7c4dbb (127.9.236.1:41083)
I20260929 19:13:24.106933 26467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9edb3e5acbee4165a87480cf3b33fbdb" candidate_uuid: "f8375ab687994ea6868fd4b1d054b00e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d"
I20260929 19:13:24.106956 26196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9edb3e5acbee4165a87480cf3b33fbdb" candidate_uuid: "f8375ab687994ea6868fd4b1d054b00e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb"
I20260929 19:13:24.107040 26196 raft_consensus.cc:3037] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:13:24.107043 26467 raft_consensus.cc:3037] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:13:24.107805 26196 raft_consensus.cc:2454] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8375ab687994ea6868fd4b1d054b00e in term 1.
I20260929 19:13:24.107889 26467 raft_consensus.cc:2454] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8375ab687994ea6868fd4b1d054b00e in term 1.
I20260929 19:13:24.108009 26267 leader_election.cc:302] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f8375ab687994ea6868fd4b1d054b00e, fc1d458bd6ff4dd39b4cb0303b7c4dbb; no voters:
I20260929 19:13:24.108116 26526 raft_consensus.cc:2781] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:13:24.108279 26526 raft_consensus.cc:686] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e [term 1 LEADER]: Becoming Leader. State: Replica: f8375ab687994ea6868fd4b1d054b00e, State: Running, Role: LEADER
I20260929 19:13:24.108412 26526 consensus_queue.cc:237] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:24.109287 26053 catalog_manager.cc:5495] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e reported cstate change: term changed from 0 to 1, leader changed from <none> to f8375ab687994ea6868fd4b1d054b00e (127.9.236.2). New cstate: current_term: 1 leader_uuid: "f8375ab687994ea6868fd4b1d054b00e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } health_report { overall_health: UNKNOWN } } }
W20260929 19:13:24.121629 26514 tablet.cc:2369] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:13:24.133927 26521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:24.131593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60040 (local address 127.9.236.2:42955)
0929 19:13:24.131667 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:24.131669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:24.131678 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:13:24.131710 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:24.131711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:24.131730 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:24.131785 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:13:24.131929 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:24.132382 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:24.133066 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:24.133282 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:24.133338 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:24.133482 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:13:24.133531 (+ 49us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0929 19:13:24.133820 (+ 289us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60040: token expired
Metrics: {"server-negotiator.queue_time_us":44}
W20260929 19:13:24.133996 26521 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60040: token expired
I20260929 19:13:24.141389 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:24.134316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51790 (local address 127.9.236.62:37539)
0929 19:13:24.134392 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:24.134394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:24.134402 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:13:24.134422 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:24.134422 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:24.134432 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:24.134477 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:13:24.135108 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:24.135765 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:24.136625 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:24.136915 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:24.136991 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:24.138428 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:13:24.138429 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:13:24.138430 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:13:24.138437 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:13:24.140179 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:24.140549 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:24.140550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:24.140550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:24.140609 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:24.140810 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:24.140811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:24.140812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:24.140941 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:13:24.141045 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:24.141196 (+ 151us) server_negotiation.cc:299] Negotiation successful
0929 19:13:24.141281 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 19:13:24.144543 26521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:24.142580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60054 (local address 127.9.236.2:42955)
0929 19:13:24.142646 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:24.142648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:24.142656 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:13:24.142673 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:24.142674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:24.142684 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:24.142734 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:13:24.142851 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:24.143267 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:24.143966 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:24.144135 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:24.144186 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:24.144283 (+ 97us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:13:24.144379 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:13:24.144418 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:24.144464 (+ 46us) server_negotiation.cc:299] Negotiation successful
0929 19:13:24.144479 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 19:13:24.146466 26196 raft_consensus.cc:1261] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Refusing update from remote peer f8375ab687994ea6868fd4b1d054b00e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:13:24.146489 26467 raft_consensus.cc:1261] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 1 FOLLOWER]: Refusing update from remote peer f8375ab687994ea6868fd4b1d054b00e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:13:24.146837 26529 consensus_queue.cc:1035] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:13:24.146974 26526 consensus_queue.cc:1035] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:13:24.148945 26532 mvcc.cc:205] Tried to move back new op lower bound from 7334744900181196800 to 7334744900028784640. Current Snapshot: MvccSnapshot[applied={T|T < 7334744900181196800}]
I20260929 19:13:24.149276 26533 mvcc.cc:205] Tried to move back new op lower bound from 7334744900181196800 to 7334744900028784640. Current Snapshot: MvccSnapshot[applied={T|T < 7334744900181196800}]
I20260929 19:13:24.151588 26053 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:13:24.151683 26053 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:13:24.152431 26053 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069: Sending DeleteTablet for 3 replicas of tablet 9edb3e5acbee4165a87480cf3b33fbdb
I20260929 19:13:24.152714 26447 tablet_service.cc:1514] Processing DeleteTablet for tablet 9edb3e5acbee4165a87480cf3b33fbdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:13:24 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:33926
I20260929 19:13:24.152758 26315 tablet_service.cc:1514] Processing DeleteTablet for tablet 9edb3e5acbee4165a87480cf3b33fbdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:13:24 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:60028
I20260929 19:13:24.152823 26174 tablet_service.cc:1514] Processing DeleteTablet for tablet 9edb3e5acbee4165a87480cf3b33fbdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:13:24 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:54578
I20260929 19:13:24.152983 26542 tablet_replica.cc:330] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: stopping tablet replica
I20260929 19:13:24.153038 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 26092
I20260929 19:13:24.153091 26542 raft_consensus.cc:2229] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:13:24.153667 26542 raft_consensus.cc:2258] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:13:24.154196 26542 ts_tablet_manager.cc:1905] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:13:24.156177 26542 ts_tablet_manager.cc:1918] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:13:24.156253 26542 log.cc:1196] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/9edb3e5acbee4165a87480cf3b33fbdb
I20260929 19:13:24.156533 26542 ts_tablet_manager.cc:1939] T 9edb3e5acbee4165a87480cf3b33fbdb P d0be3da1d3f24a4ab5a5c5dd9ac1b53d: Deleting consensus metadata
I20260929 19:13:24.156966 26041 catalog_manager.cc:4841] TS d0be3da1d3f24a4ab5a5c5dd9ac1b53d (127.9.236.3:35887): tablet 9edb3e5acbee4165a87480cf3b33fbdb (table test-table [id=abcbf14f7b044a51ba7993bcfad61c9e]) successfully deleted
I20260929 19:13:24.157207 26543 tablet_replica.cc:330] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e: stopping tablet replica
I20260929 19:13:24.157313 26543 raft_consensus.cc:2229] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:13:24.157433 26543 raft_consensus.cc:2258] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:13:24.157886 26543 ts_tablet_manager.cc:1905] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:13:24.159582 26543 ts_tablet_manager.cc:1918] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:13:24.159641 26543 log.cc:1196] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/9edb3e5acbee4165a87480cf3b33fbdb
I20260929 19:13:24.159837 26543 ts_tablet_manager.cc:1939] T 9edb3e5acbee4165a87480cf3b33fbdb P f8375ab687994ea6868fd4b1d054b00e: Deleting consensus metadata
I20260929 19:13:24.160225 26040 catalog_manager.cc:4841] TS f8375ab687994ea6868fd4b1d054b00e (127.9.236.2:42955): tablet 9edb3e5acbee4165a87480cf3b33fbdb (table test-table [id=abcbf14f7b044a51ba7993bcfad61c9e]) successfully deleted
W20260929 19:13:24.160964 26038 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.236.1:41083 (error 108)
I20260929 19:13:24.161449 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 26248
I20260929 19:13:24.161561 26519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:24.161407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:41083 (local address 127.0.0.1:54594)
0929 19:13:24.161473 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:24.161519 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.1:41083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260929 19:13:24.162415 26038 catalog_manager.cc:4584] TS fc1d458bd6ff4dd39b4cb0303b7c4dbb (127.9.236.1:41083): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9edb3e5acbee4165a87480cf3b33fbdb: Network error: Client connection negotiation failed: client connection to 127.9.236.1:41083: connect: Connection refused (error 111)
I20260929 19:13:24.169981 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 26384
I20260929 19:13:24.174077 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 26021
I20260929 19:13:44.180223 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:37539
--webserver_interface=127.9.236.62
--webserver_port=40689
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:37539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:13:44.268379 26546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:13:44.268632 26546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:13:44.268656 26546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:13:44.269923 26546 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 19:13:44.269966 26546 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 19:13:44.269979 26546 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 19:13:44.269989 26546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:13:44.270001 26546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:13:44.270038 26546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:13:44.270066 26546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:13:44.270092 26546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:13:44.271653 26546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:37539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:37539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=40689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:13:44.271922 26546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:13:44.272164 26546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:13:44.277642 26546 server_base.cc:1034] running on GCE node
W20260929 19:13:44.277580 26554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:13:44.277563 26551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:13:44.277546 26552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:13:44.278136 26546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:13:44.278455 26546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:13:44.279595 26546 hybrid_clock.cc:648] HybridClock initialized: now 1790709224279583 us; error 36 us; skew 500 ppm
Sep 29 19:13:44 dist-test-slave-wftq krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709224, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:13:44.282199 26546 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:13:44.282629 26546 webserver.cc:460] Webserver started at http://127.9.236.62:40689/ using document root <none> and password file <none>
I20260929 19:13:44.282852 26546 fs_manager.cc:359] Metadata directory not provided
I20260929 19:13:44.282902 26546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:13:44.284356 26546 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:13:44.285048 26561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:44.285260 26546 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:13:44.285346 26546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "b9850d4c2a4043cd86865de232d53069"
format_stamp: "Formatted at 2026-09-29 19:13:02 on dist-test-slave-wftq"
server_key: "acd5f8dacf21040a545707bdda8da682"
server_key_iv: "6a43419585487c1426b34cf37f36b63d"
server_key_version: "encryptionkey@0"
I20260929 19:13:44.285718 26546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:13:44.296001 26546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:13:44.297896 26546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:13:44.298027 26546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:13:44.302120 26546 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:37539
I20260929 19:13:44.302184 26613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:37539 every 8 connection(s)
I20260929 19:13:44.302532 26546 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:13:44.304350 26614 sys_catalog.cc:263] Verifying existing consensus state
I20260929 19:13:44.305066 26614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069: Bootstrap starting.
I20260929 19:13:44.305945 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 26546
I20260929 19:13:44.306291 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:41083
--local_ip_for_outbound_sockets=127.9.236.1
--tserver_master_addrs=127.9.236.62:37539
--webserver_port=43677
--webserver_interface=127.9.236.1
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:13:44.315953 26614 log.cc:824] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069: Log is configured to *not* fsync() on all Append() calls
I20260929 19:13:44.318677 26614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:13:44.319029 26614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069: Bootstrap complete.
I20260929 19:13:44.321534 26614 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9850d4c2a4043cd86865de232d53069" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37539 } }
I20260929 19:13:44.321802 26614 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9850d4c2a4043cd86865de232d53069, State: Initialized, Role: FOLLOWER
I20260929 19:13:44.321985 26614 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9850d4c2a4043cd86865de232d53069" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37539 } }
I20260929 19:13:44.322064 26614 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:13:44.322109 26614 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:13:44.322162 26614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 2 FOLLOWER]: Advancing to term 3
I20260929 19:13:44.323316 26614 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9850d4c2a4043cd86865de232d53069" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37539 } }
I20260929 19:13:44.323429 26614 leader_election.cc:302] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9850d4c2a4043cd86865de232d53069; no voters:
I20260929 19:13:44.323625 26614 leader_election.cc:288] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 19:13:44.323693 26618 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 19:13:44.323846 26618 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [term 3 LEADER]: Becoming Leader. State: Replica: b9850d4c2a4043cd86865de232d53069, State: Running, Role: LEADER
I20260929 19:13:44.323977 26614 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:13:44.323949 26618 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9850d4c2a4043cd86865de232d53069" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37539 } }
I20260929 19:13:44.324429 26619 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b9850d4c2a4043cd86865de232d53069" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9850d4c2a4043cd86865de232d53069" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37539 } } }
I20260929 19:13:44.324540 26619 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [sys.catalog]: This master's current role is: LEADER
I20260929 19:13:44.324463 26620 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9850d4c2a4043cd86865de232d53069. Latest consensus state: current_term: 3 leader_uuid: "b9850d4c2a4043cd86865de232d53069" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9850d4c2a4043cd86865de232d53069" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 37539 } } }
I20260929 19:13:44.324610 26620 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069 [sys.catalog]: This master's current role is: LEADER
I20260929 19:13:44.324795 26625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:13:44.325806 26625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:13:44.325924 26625 catalog_manager.cc:1259] Loaded cluster ID: 3a30a93c8b4045a6a4616894898973c9
I20260929 19:13:44.325966 26625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:13:44.327136 26625 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:13:44.327332 26625 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069: Loaded TSK: 0
I20260929 19:13:44.327387 26625 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b9850d4c2a4043cd86865de232d53069: Loaded TSK: 1
F20260929 19:13:44.327638 26625 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fde7a09b9ad google::LogMessage::Fail() at ??:0
@ 0x7fde7a09e88a google::LogMessage::SendToLog() at ??:0
@ 0x7fde7a09b4de google::LogMessage::Flush() at ??:0
@ 0x7fde7a09ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fde7c808104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fde7c805d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fde81ff51cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fde81ff548e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fde81ff5594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fde81fbbc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fde81fc5ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fde7bf690d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fde7bf5e954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fde7cdc26db start_thread at ??:0
@ 0x7fde78b9961f clone at ??:0
W20260929 19:13:44.400127 26616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:13:44.400339 26616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:13:44.400372 26616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:13:44.401535 26616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:13:44.401592 26616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:13:44.401623 26616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:13:44.403096 26616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:41083
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=43677
--tserver_master_addrs=127.9.236.62:37539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:13:44.403317 26616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:13:44.403537 26616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:13:44.408542 26655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:13:44.408600 26656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:13:44.408557 26658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:13:44.408914 26616 server_base.cc:1034] running on GCE node
I20260929 19:13:44.409132 26616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:13:44.409370 26616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:13:44.410531 26616 hybrid_clock.cc:648] HybridClock initialized: now 1790709224410503 us; error 43 us; skew 500 ppm
Sep 29 19:13:44 dist-test-slave-wftq krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709224, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:13:44.413280 26616 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:13:44.413726 26616 webserver.cc:460] Webserver started at http://127.9.236.1:43677/ using document root <none> and password file <none>
I20260929 19:13:44.413933 26616 fs_manager.cc:359] Metadata directory not provided
I20260929 19:13:44.413981 26616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:13:44.415282 26616 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:13:44.415982 26665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:44.416260 26616 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:13:44.416350 26616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb"
format_stamp: "Formatted at 2026-09-29 19:13:02 on dist-test-slave-wftq"
server_key: "16eedb70479a5c26aa6127febe4b36dc"
server_key_iv: "09c088a63334f424604990ff1da0f2d1"
server_key_version: "encryptionkey@0"
I20260929 19:13:44.416693 26616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:13:44.431264 26616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:13:44.433562 26616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:13:44.433679 26616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:13:44.433899 26616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:13:44.434384 26672 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 19:13:44.436678 26616 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 19:13:44.436738 26616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260929 19:13:44.436772 26616 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 19:13:44.437546 26616 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 19:13:44.437578 26616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260929 19:13:44.437697 26672 tablet_bootstrap.cc:492] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Bootstrap starting.
I20260929 19:13:44.444384 26616 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:41083
I20260929 19:13:44.444564 26779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:41083 every 8 connection(s)
I20260929 19:13:44.444808 26616 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:13:44.450528 26672 log.cc:824] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Log is configured to *not* fsync() on all Append() calls
I20260929 19:13:44.453039 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 26616
I20260929 19:13:44.453329 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:42955
--local_ip_for_outbound_sockets=127.9.236.2
--tserver_master_addrs=127.9.236.62:37539
--webserver_port=34799
--webserver_interface=127.9.236.2
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:13:44.453780 26672 tablet_bootstrap.cc:492] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 19:13:44.454145 26672 tablet_bootstrap.cc:492] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Bootstrap complete.
I20260929 19:13:44.455331 26672 ts_tablet_manager.cc:1397] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.010s
I20260929 19:13:44.456939 26672 raft_consensus.cc:348] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:44.457365 26672 raft_consensus.cc:729] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1d458bd6ff4dd39b4cb0303b7c4dbb, State: Initialized, Role: FOLLOWER
I20260929 19:13:44.457559 26672 consensus_queue.cc:260] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:44.457895 26672 ts_tablet_manager.cc:1428] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260929 19:13:44.534869 26782 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:13:44.445438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.1:49553)
0929 19:13:44.445610 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:44.445650 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 19:13:44.446104 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:13:44.534544 (+ 88440us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":79,"threads_started":1}
W20260929 19:13:44.535250 26780 heartbeater.cc:644] Failed to heartbeat to 127.9.236.62:37539 (0 consecutive failures): Network error: Failed to ping master at 127.9.236.62:37539: Client connection negotiation failed: client connection to 127.9.236.62:37539: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 19:13:44.535637 26782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:44.535516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.1:38165)
0929 19:13:44.535572 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:44.535610 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260929 19:13:44.535954 26782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:44.535876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.1:37357)
0929 19:13:44.535912 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:44.535933 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260929 19:13:44.535861 26784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:13:44.536072 26784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:13:44.536119 26784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:13:44.536124 26780 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 19:13:44.537499 26784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:13:44.537554 26784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:13:44.537581 26784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:13:44.539117 26784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:42955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=34799
--tserver_master_addrs=127.9.236.62:37539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:13:44.539337 26784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:13:44.539553 26784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:13:44.544507 26791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:13:44.544601 26793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:13:44.544510 26790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:13:44.544934 26784 server_base.cc:1034] running on GCE node
I20260929 19:13:44.545174 26784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:13:44.545433 26784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:13:44.546603 26784 hybrid_clock.cc:648] HybridClock initialized: now 1790709224546587 us; error 39 us; skew 500 ppm
Sep 29 19:13:44 dist-test-slave-wftq krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709224, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:13:44.549587 26784 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:13:44.550079 26784 webserver.cc:460] Webserver started at http://127.9.236.2:34799/ using document root <none> and password file <none>
I20260929 19:13:44.550278 26784 fs_manager.cc:359] Metadata directory not provided
I20260929 19:13:44.550329 26784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:13:44.551723 26784 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:13:44.552318 26800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:44.552495 26784 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 19:13:44.552589 26784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "f8375ab687994ea6868fd4b1d054b00e"
format_stamp: "Formatted at 2026-09-29 19:13:02 on dist-test-slave-wftq"
server_key: "ce9fc6758ea6e61a1be921381b9ee3de"
server_key_iv: "6ef94e67ffc09c1d95234a67f774743d"
server_key_version: "encryptionkey@0"
I20260929 19:13:44.552985 26784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:13:44.578212 26784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:13:44.580286 26784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:13:44.580422 26784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:13:44.580626 26784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:13:44.580976 26784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:13:44.581015 26784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:44.581039 26784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:13:44.581050 26784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:44.587873 26784 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:42955
I20260929 19:13:44.587941 26913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:42955 every 8 connection(s)
I20260929 19:13:44.588328 26784 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:13:44.589511 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:44.589076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.2:37233)
0929 19:13:44.589281 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:44.589365 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":91,"threads_started":1}
W20260929 19:13:44.589937 26914 heartbeater.cc:644] Failed to heartbeat to 127.9.236.62:37539 (0 consecutive failures): Network error: Failed to ping master at 127.9.236.62:37539: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
I20260929 19:13:44.590224 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:44.590125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.2:42671)
0929 19:13:44.590176 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:44.590206 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 19:13:44.590376 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 26784
I20260929 19:13:44.590564 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:44.590484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.2:54229)
0929 19:13:44.590518 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:44.590541 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260929 19:13:44.590653 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:35887
--local_ip_for_outbound_sockets=127.9.236.3
--tserver_master_addrs=127.9.236.62:37539
--webserver_port=42627
--webserver_interface=127.9.236.3
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:45117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:13:44.590668 26914 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 19:13:44.679056 26917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:13:44.679281 26917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:13:44.679303 26917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:13:44.680465 26917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:13:44.680510 26917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:13:44.680537 26917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:13:44.682076 26917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:45117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:35887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=42627
--tserver_master_addrs=127.9.236.62:37539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:13:44.682314 26917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:13:44.682559 26917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:13:44.688092 26925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:13:44.688144 26923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:13:44.688026 26922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:13:44.688410 26917 server_base.cc:1034] running on GCE node
I20260929 19:13:44.688655 26917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:13:44.688923 26917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:13:44.690065 26917 hybrid_clock.cc:648] HybridClock initialized: now 1790709224690064 us; error 57 us; skew 500 ppm
Sep 29 19:13:44 dist-test-slave-wftq krb5kdc[25474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709224, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:13:44.692552 26917 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:13:44.693022 26917 webserver.cc:460] Webserver started at http://127.9.236.3:42627/ using document root <none> and password file <none>
I20260929 19:13:44.693286 26917 fs_manager.cc:359] Metadata directory not provided
I20260929 19:13:44.693341 26917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:13:44.694792 26917 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:13:44.695711 26932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:44.695925 26917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:13:44.696005 26917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d"
format_stamp: "Formatted at 2026-09-29 19:13:03 on dist-test-slave-wftq"
server_key: "0322d575d2c7ea48b37db87f083be7d1"
server_key_iv: "59a8347e8ad657f3f6378aa4fd70b23f"
server_key_version: "encryptionkey@0"
I20260929 19:13:44.696384 26917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:13:44.713693 26917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:13:44.715613 26917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:13:44.715734 26917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:13:44.715965 26917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:13:44.716267 26917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:13:44.716305 26917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:44.716325 26917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:13:44.716353 26917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:13:44.722765 26917 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:35887
I20260929 19:13:44.722857 27045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:35887 every 8 connection(s)
I20260929 19:13:44.723163 26917 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:13:44.724310 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:44.723895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.3:47407)
0929 19:13:44.724094 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:44.724170 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":104,"threads_started":1}
W20260929 19:13:44.724740 27046 heartbeater.cc:644] Failed to heartbeat to 127.9.236.62:37539 (0 consecutive failures): Network error: Failed to ping master at 127.9.236.62:37539: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
I20260929 19:13:44.725016 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:44.724925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.3:50985)
0929 19:13:44.724971 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:44.724999 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260929 19:13:44.725279 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:44.725204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.3:45771)
0929 19:13:44.725243 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:44.725266 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 19:13:44.725387 27046 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 19:13:44.726400 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 26917
I20260929 19:13:45.536828 27050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:45.536525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.1:47861)
0929 19:13:45.536679 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:45.536740 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":64,"threads_started":1}
I20260929 19:13:45.591253 27051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:45.590977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.2:39859)
0929 19:13:45.591122 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:45.591174 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":57,"threads_started":1}
I20260929 19:13:45.725986 27052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:45.725713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.3:59165)
0929 19:13:45.725865 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:45.725914 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":67,"threads_started":1}
I20260929 19:13:45.934768 27053 raft_consensus.cc:482] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:13:45.934962 27053 raft_consensus.cc:504] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:45.935350 27053 leader_election.cc:288] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d0be3da1d3f24a4ab5a5c5dd9ac1b53d (127.9.236.3:35887), f8375ab687994ea6868fd4b1d054b00e (127.9.236.2:42955)
Sep 29 19:13:45 dist-test-slave-wftq krb5kdc[25474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709224, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.3@KRBTEST.COM
Sep 29 19:13:45 dist-test-slave-wftq krb5kdc[25474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709224, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.2@KRBTEST.COM
I20260929 19:13:45.946000 27056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:45.935679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:58317 (local address 127.9.236.2:42955)
0929 19:13:45.935895 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:45.935905 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:45.935937 (+ 32us) server_negotiation.cc:407] Connection header received
0929 19:13:45.936481 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:45.936483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:45.936536 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:45.936693 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:13:45.937452 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:45.938337 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:45.939487 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:45.939893 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:45.940002 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:45.942780 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:13:45.942783 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:13:45.942784 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:13:45.942794 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:13:45.944403 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:45.944829 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:45.944831 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:45.944832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:45.944914 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:45.945150 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:45.945151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:45.945151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:45.945385 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:13:45.945452 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:45.945761 (+ 309us) server_negotiation.cc:299] Negotiation successful
0929 19:13:45.945877 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":54,"threads_started":1}
I20260929 19:13:45.945988 27050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:45.935494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:35887 (local address 127.9.236.1:33443)
0929 19:13:45.935598 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:45.935612 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:13:45.935686 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:13:45.936632 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:13:45.936635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:13:45.937068 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:13:45.937322 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:45.937330 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:45.938452 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:45.938453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:45.939399 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:45.939401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:45.939961 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:45.939961 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:45.940080 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:45.940697 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:13:45.940702 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:13:45.942284 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:13:45.944345 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:13:45.944351 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:13:45.944354 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:13:45.944643 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:13:45.944937 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:13:45.944938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:13:45.944938 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:13:45.945049 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:13:45.945478 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:13:45.945479 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:13:45.945591 (+ 112us) client_negotiation.cc:769] Sending connection context
0929 19:13:45.945680 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 19:13:45.945889 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":75}
I20260929 19:13:45.946028 27055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:45.935633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:33443 (local address 127.9.236.3:35887)
0929 19:13:45.935843 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0929 19:13:45.935855 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 19:13:45.935889 (+ 34us) server_negotiation.cc:407] Connection header received
0929 19:13:45.936371 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:13:45.936374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:13:45.936420 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:13:45.936545 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:13:45.937443 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:45.938346 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:45.939490 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:13:45.939875 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:13:45.939972 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:45.942397 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:13:45.942405 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:13:45.942405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:13:45.942417 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:13:45.944269 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:45.944742 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:45.944746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:45.944746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:45.944867 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:13:45.945168 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:13:45.945169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:13:45.945170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:13:45.945401 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:13:45.945485 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 19:13:45.945763 (+ 278us) server_negotiation.cc:299] Negotiation successful
0929 19:13:45.945920 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":79,"threads_started":1}
I20260929 19:13:45.946101 27054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:45.935586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:42955 (local address 127.9.236.1:58317)
0929 19:13:45.935771 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:45.935779 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 19:13:45.935816 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:13:45.936701 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:13:45.936703 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:13:45.937076 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:13:45.937322 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:45.937331 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:45.938448 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:45.938449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:45.939400 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:13:45.939403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:13:45.939976 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:13:45.939977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:13:45.940077 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:13:45.940646 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:13:45.940654 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:13:45.942689 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:13:45.944511 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:13:45.944512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:13:45.944513 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:13:45.944742 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:13:45.944989 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:13:45.944989 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:13:45.944989 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:13:45.945063 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:13:45.945460 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:13:45.945467 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:13:45.945591 (+ 124us) client_negotiation.cc:769] Sending connection context
0929 19:13:45.945682 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 19:13:45.945976 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":196,"thread_start_us":81,"threads_started":1}
W20260929 19:13:45.946821 26669 leader_election.cc:341] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d0be3da1d3f24a4ab5a5c5dd9ac1b53d (127.9.236.3:35887): Not found: Tablet not found: 9edb3e5acbee4165a87480cf3b33fbdb
W20260929 19:13:45.947451 26668 leader_election.cc:341] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f8375ab687994ea6868fd4b1d054b00e (127.9.236.2:42955): Not found: Tablet not found: 9edb3e5acbee4165a87480cf3b33fbdb
I20260929 19:13:45.947497 26668 leader_election.cc:302] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc1d458bd6ff4dd39b4cb0303b7c4dbb; no voters: d0be3da1d3f24a4ab5a5c5dd9ac1b53d, f8375ab687994ea6868fd4b1d054b00e
I20260929 19:13:45.947611 27053 raft_consensus.cc:2726] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:13:46.537631 27057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:46.537377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.1:52411)
0929 19:13:46.537525 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:46.537568 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":66,"threads_started":1}
I20260929 19:13:46.592000 27058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:46.591726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.2:52545)
0929 19:13:46.591894 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:46.591947 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":73,"threads_started":1}
I20260929 19:13:46.726681 27059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:46.726435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.3:44457)
0929 19:13:46.726592 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:46.726631 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
I20260929 19:13:47.538580 27060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:47.538312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.1:54965)
0929 19:13:47.538492 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:47.538528 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":106,"threads_started":1}
I20260929 19:13:47.592715 27061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:47.592431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.2:52791)
0929 19:13:47.592588 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:47.592656 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":66,"threads_started":1}
I20260929 19:13:47.727347 27062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:47.727075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.3:57615)
0929 19:13:47.727245 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:47.727287 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":62,"threads_started":1}
I20260929 19:13:48.113207 27063 raft_consensus.cc:482] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:13:48.113348 27063 raft_consensus.cc:504] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:48.113566 27063 leader_election.cc:288] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d0be3da1d3f24a4ab5a5c5dd9ac1b53d (127.9.236.3:35887), f8375ab687994ea6868fd4b1d054b00e (127.9.236.2:42955)
W20260929 19:13:48.114058 26669 leader_election.cc:341] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d0be3da1d3f24a4ab5a5c5dd9ac1b53d (127.9.236.3:35887): Not found: Tablet not found: 9edb3e5acbee4165a87480cf3b33fbdb
W20260929 19:13:48.114144 26668 leader_election.cc:341] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f8375ab687994ea6868fd4b1d054b00e (127.9.236.2:42955): Not found: Tablet not found: 9edb3e5acbee4165a87480cf3b33fbdb
I20260929 19:13:48.114228 26668 leader_election.cc:302] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc1d458bd6ff4dd39b4cb0303b7c4dbb; no voters: d0be3da1d3f24a4ab5a5c5dd9ac1b53d, f8375ab687994ea6868fd4b1d054b00e
I20260929 19:13:48.114343 27063 raft_consensus.cc:2726] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:13:48.539392 27064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:48.539130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.1:56371)
0929 19:13:48.539285 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:48.539341 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":87,"threads_started":1}
I20260929 19:13:48.593420 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:48.593143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.2:41247)
0929 19:13:48.593294 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:48.593341 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":65,"threads_started":1}
I20260929 19:13:48.728080 27066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:48.727823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.3:34285)
0929 19:13:48.727999 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:48.728039 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":80,"threads_started":1}
I20260929 19:13:49.540297 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:49.539983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.1:59045)
0929 19:13:49.540187 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:49.540247 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":96,"threads_started":1}
I20260929 19:13:49.594205 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:49.593923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.2:55525)
0929 19:13:49.594112 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:49.594159 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":97,"threads_started":1}
I20260929 19:13:49.728971 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:49.728701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.3:59355)
0929 19:13:49.728848 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:49.728904 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":62,"threads_started":1}
I20260929 19:13:50.541273 27070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:50.540888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.1:47163)
0929 19:13:50.541134 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:50.541205 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":89,"threads_started":1}
I20260929 19:13:50.594944 27071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:50.594660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.2:56821)
0929 19:13:50.594839 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:50.594894 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":74,"threads_started":1}
I20260929 19:13:50.729761 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:50.729478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.3:48723)
0929 19:13:50.729647 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:50.729707 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":76,"threads_started":1}
I20260929 19:13:50.966682 27073 raft_consensus.cc:482] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:13:50.966821 27073 raft_consensus.cc:504] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:50.967010 27073 leader_election.cc:288] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d0be3da1d3f24a4ab5a5c5dd9ac1b53d (127.9.236.3:35887), f8375ab687994ea6868fd4b1d054b00e (127.9.236.2:42955)
W20260929 19:13:50.967494 26668 leader_election.cc:341] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f8375ab687994ea6868fd4b1d054b00e (127.9.236.2:42955): Not found: Tablet not found: 9edb3e5acbee4165a87480cf3b33fbdb
W20260929 19:13:50.967599 26669 leader_election.cc:341] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d0be3da1d3f24a4ab5a5c5dd9ac1b53d (127.9.236.3:35887): Not found: Tablet not found: 9edb3e5acbee4165a87480cf3b33fbdb
I20260929 19:13:50.967653 26669 leader_election.cc:302] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc1d458bd6ff4dd39b4cb0303b7c4dbb; no voters: d0be3da1d3f24a4ab5a5c5dd9ac1b53d, f8375ab687994ea6868fd4b1d054b00e
I20260929 19:13:50.967752 27073 raft_consensus.cc:2726] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:13:51.542203 27074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:51.541872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.1:58575)
0929 19:13:51.542088 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:51.542150 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":86,"threads_started":1}
I20260929 19:13:51.595757 27075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:51.595469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.2:49327)
0929 19:13:51.595630 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:51.595690 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":68,"threads_started":1}
I20260929 19:13:51.730643 27076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:51.730341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.3:33569)
0929 19:13:51.730516 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:51.730576 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":63,"threads_started":1}
I20260929 19:13:52.543112 27077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:52.542809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.1:36353)
0929 19:13:52.542988 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:52.543044 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":71,"threads_started":1}
I20260929 19:13:52.596560 27078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:52.596306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.2:46981)
0929 19:13:52.596456 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:52.596508 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260929 19:13:52.731406 27079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:52.731138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.3:60379)
0929 19:13:52.731307 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:52.731350 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":67,"threads_started":1}
I20260929 19:13:53.544075 27080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:53.543750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.1:36777)
0929 19:13:53.543971 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:53.544012 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":110,"threads_started":1}
I20260929 19:13:53.597306 27081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:53.597041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.2:56209)
0929 19:13:53.597194 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:53.597243 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
I20260929 19:13:53.732292 27082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:53.732041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.3:49549)
0929 19:13:53.732197 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:53.732244 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":76,"threads_started":1}
I20260929 19:13:54.181149 27083 raft_consensus.cc:482] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:13:54.181277 27083 raft_consensus.cc:504] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0be3da1d3f24a4ab5a5c5dd9ac1b53d" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35887 } } peers { permanent_uuid: "f8375ab687994ea6868fd4b1d054b00e" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 42955 } } peers { permanent_uuid: "fc1d458bd6ff4dd39b4cb0303b7c4dbb" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41083 } }
I20260929 19:13:54.181496 27083 leader_election.cc:288] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d0be3da1d3f24a4ab5a5c5dd9ac1b53d (127.9.236.3:35887), f8375ab687994ea6868fd4b1d054b00e (127.9.236.2:42955)
W20260929 19:13:54.181882 26668 leader_election.cc:341] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f8375ab687994ea6868fd4b1d054b00e (127.9.236.2:42955): Not found: Tablet not found: 9edb3e5acbee4165a87480cf3b33fbdb
W20260929 19:13:54.182024 26669 leader_election.cc:341] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d0be3da1d3f24a4ab5a5c5dd9ac1b53d (127.9.236.3:35887): Not found: Tablet not found: 9edb3e5acbee4165a87480cf3b33fbdb
I20260929 19:13:54.182072 26669 leader_election.cc:302] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc1d458bd6ff4dd39b4cb0303b7c4dbb; no voters: d0be3da1d3f24a4ab5a5c5dd9ac1b53d, f8375ab687994ea6868fd4b1d054b00e
I20260929 19:13:54.182170 27083 raft_consensus.cc:2726] T 9edb3e5acbee4165a87480cf3b33fbdb P fc1d458bd6ff4dd39b4cb0303b7c4dbb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 19:13:54.545013 27084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:54.544673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.1:44035)
0929 19:13:54.544867 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:54.544933 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":75,"threads_started":1}
I20260929 19:13:54.598063 27085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:54.597783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.2:57607)
0929 19:13:54.597945 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:54.598000 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
I20260929 19:13:54.733070 27086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:54.732800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.3:34285)
0929 19:13:54.732957 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:54.733006 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":64,"threads_started":1}
I20260929 19:13:55.545855 27087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:55.545587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.1:57273)
0929 19:13:55.545741 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:55.545791 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":63,"threads_started":1}
I20260929 19:13:55.598826 27088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:55.598565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.2:57041)
0929 19:13:55.598742 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:55.598783 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":87,"threads_started":1}
I20260929 19:13:55.733877 27089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:55.733608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.3:42845)
0929 19:13:55.733753 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:55.733809 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":66,"threads_started":1}
I20260929 19:13:56.546746 27090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:56.546503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.1:58903)
0929 19:13:56.546655 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:56.546694 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260929 19:13:56.599588 27091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:56.599332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.2:52821)
0929 19:13:56.599482 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:56.599527 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":68,"threads_started":1}
I20260929 19:13:56.734611 27092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:56.734330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.3:35605)
0929 19:13:56.734501 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:56.734549 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":80,"threads_started":1}
I20260929 19:13:57.547670 27093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:57.547405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.1:40075)
0929 19:13:57.547556 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:57.547598 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":80,"threads_started":1}
I20260929 19:13:57.600383 27094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:57.600115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.2:51469)
0929 19:13:57.600259 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:57.600318 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":65,"threads_started":1}
I20260929 19:13:57.735550 27095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:57.735248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.3:34015)
0929 19:13:57.735433 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:57.735487 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":94,"threads_started":1}
I20260929 19:13:58.548398 27096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:58.548149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.1:59461)
0929 19:13:58.548309 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:58.548349 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":78,"threads_started":1}
I20260929 19:13:58.601161 27097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:58.600888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.2:53227)
0929 19:13:58.601021 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:58.601072 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
I20260929 19:13:58.736331 27098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:58.736111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.3:35873)
0929 19:13:58.736247 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:58.736282 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
I20260929 19:13:59.549193 27099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:59.548883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.1:45087)
0929 19:13:59.549052 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:59.549096 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":61,"threads_started":1}
I20260929 19:13:59.601866 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:59.601629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.2:46443)
0929 19:13:59.601781 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:59.601818 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260929 19:13:59.727775 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 26616
I20260929 19:13:59.732400 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 26784
I20260929 19:13:59.737182 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 26917
I20260929 19:13:59.737217 27101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:13:59.736961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:37539 (local address 127.9.236.3:59757)
0929 19:13:59.737123 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 19:13:59.737162 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.62:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":65,"threads_started":1}
2026-09-29T19:13:59Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58380 ms)
[----------] 30 tests from SecurityITest (231224 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:13:59 dist-test-slave-wftq krb5kdc[27104](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:13:59 dist-test-slave-wftq krb5kdc[27104](info): set up 2 sockets
Sep 29 19:13:59 dist-test-slave-wftq krb5kdc[27104](info): commencing operation
krb5kdc: starting...
W20260929 19:14:00.696695 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:00 dist-test-slave-wftq krb5kdc[27104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:14:00.816051 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:35523
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:34367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:35523
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:00.897938 27117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:00.898151 27117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:00.898180 27117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:00.899307 27117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:00.899348 27117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:00.899366 27117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:00.899382 27117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:00.899396 27117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:00.900821 27117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:34367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:35523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:35523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:00.901052 27117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:00.901278 27117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:14:00.906418 27117 server_base.cc:1034] running on GCE node
W20260929 19:14:00.906376 27123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:00.906397 27122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:00.906380 27125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:00.906804 27117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:00.907104 27117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:00.908257 27117 hybrid_clock.cc:648] HybridClock initialized: now 1790709240908233 us; error 48 us; skew 500 ppm
Sep 29 19:14:00 dist-test-slave-wftq krb5kdc[27104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709240, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:00.911124 27117 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:14:00.911666 27117 webserver.cc:460] Webserver started at http://127.9.236.62:43293/ using document root <none> and password file <none>
I20260929 19:14:00.911907 27117 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:00.911962 27117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:00.912106 27117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:00.913398 27117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "bcba2cfc1ddc4357acb3c721f53cca90"
format_stamp: "Formatted at 2026-09-29 19:14:00 on dist-test-slave-wftq"
server_key: "426db4595056c7cc84484a38a6746d95"
server_key_iv: "efaaf9116ebf694366bf13b0fe5131f3"
server_key_version: "encryptionkey@0"
I20260929 19:14:00.913803 27117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "bcba2cfc1ddc4357acb3c721f53cca90"
format_stamp: "Formatted at 2026-09-29 19:14:00 on dist-test-slave-wftq"
server_key: "426db4595056c7cc84484a38a6746d95"
server_key_iv: "efaaf9116ebf694366bf13b0fe5131f3"
server_key_version: "encryptionkey@0"
I20260929 19:14:00.915333 27117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:14:00.916225 27132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:00.916469 27117 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:00.916560 27117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "bcba2cfc1ddc4357acb3c721f53cca90"
format_stamp: "Formatted at 2026-09-29 19:14:00 on dist-test-slave-wftq"
server_key: "426db4595056c7cc84484a38a6746d95"
server_key_iv: "efaaf9116ebf694366bf13b0fe5131f3"
server_key_version: "encryptionkey@0"
I20260929 19:14:00.916635 27117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:00.926028 27117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:00.928081 27117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:00.928210 27117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:00.932726 27117 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:35523
I20260929 19:14:00.932796 27184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:35523 every 8 connection(s)
I20260929 19:14:00.933154 27117 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:00.934044 27185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:00.936267 27185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90: Bootstrap starting.
I20260929 19:14:00.936780 27185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:00.937000 27185 log.cc:824] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:00.937738 27185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90: No bootstrap required, opened a new log
I20260929 19:14:00.939244 27185 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcba2cfc1ddc4357acb3c721f53cca90" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35523 } }
I20260929 19:14:00.939379 27185 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:00.939399 27185 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcba2cfc1ddc4357acb3c721f53cca90, State: Initialized, Role: FOLLOWER
I20260929 19:14:00.939502 27185 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcba2cfc1ddc4357acb3c721f53cca90" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35523 } }
I20260929 19:14:00.939554 27185 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:00.939581 27185 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:00.939618 27185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:00.940093 27185 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcba2cfc1ddc4357acb3c721f53cca90" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35523 } }
I20260929 19:14:00.940182 27185 leader_election.cc:302] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcba2cfc1ddc4357acb3c721f53cca90; no voters:
I20260929 19:14:00.940346 27185 leader_election.cc:288] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:00.940393 27188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:00.940601 27188 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90 [term 1 LEADER]: Becoming Leader. State: Replica: bcba2cfc1ddc4357acb3c721f53cca90, State: Running, Role: LEADER
I20260929 19:14:00.940583 27185 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:00.940783 27188 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcba2cfc1ddc4357acb3c721f53cca90" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35523 } }
I20260929 19:14:00.941339 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 27117
I20260929 19:14:00.941366 27190 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcba2cfc1ddc4357acb3c721f53cca90. Latest consensus state: current_term: 1 leader_uuid: "bcba2cfc1ddc4357acb3c721f53cca90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcba2cfc1ddc4357acb3c721f53cca90" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35523 } } }
I20260929 19:14:00.941450 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:00.941462 27190 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:00.941538 27189 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcba2cfc1ddc4357acb3c721f53cca90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcba2cfc1ddc4357acb3c721f53cca90" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35523 } } }
I20260929 19:14:00.941610 27189 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:00.941658 10160 external_mini_cluster.cc:1428] Setting key 68479e737a7cede6ae6260128c5e47bf
I20260929 19:14:00.941948 27197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:00.942564 27197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:00.945338 27197 catalog_manager.cc:1347] Generated new cluster ID: dcbc69581e734450ab7750c8b117ae94
I20260929 19:14:00.945403 27197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:14:00 dist-test-slave-wftq krb5kdc[27104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:14:00.954284 27206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:00.942359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50376 (local address 127.9.236.62:35523)
0929 19:14:00.942684 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:00.942694 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:00.942736 (+ 42us) server_negotiation.cc:407] Connection header received
0929 19:14:00.942852 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:00.942860 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:00.942915 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:00.943048 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:00.943973 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:00.945183 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:00.946372 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:00.947158 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:00.947261 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:00.950420 (+ 3159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:00.950425 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:00.950426 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:00.950440 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:00.952425 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:00.952908 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:00.952911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:00.952911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:00.953025 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:00.953252 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:00.953253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:00.953253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:00.953590 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:00.953739 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:00.953870 (+ 131us) server_negotiation.cc:299] Negotiation successful
0929 19:14:00.954027 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":90,"threads_started":1}
I20260929 19:14:00.966491 27197 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:00.967077 27197 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:00.970940 27197 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90: Generated new TSK 0
I20260929 19:14:00.971187 27197 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.236.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
I20260929 19:14:01.000180 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:35523
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:34367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:01.087025 27211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:01.087335 27211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:01.087373 27211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:01.088963 27211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:01.089018 27211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:01.089057 27211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:14:01.091303 27211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:34367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:35523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:01.091745 27211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:01.092046 27211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:01.098172 27219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:01.098177 27217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:01.098178 27216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:01.099287 27211 server_base.cc:1034] running on GCE node
I20260929 19:14:01.099545 27211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:01.099892 27211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:01.101034 27211 hybrid_clock.cc:648] HybridClock initialized: now 1790709241101015 us; error 26 us; skew 500 ppm
Sep 29 19:14:01 dist-test-slave-wftq krb5kdc[27104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709241, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:01.103724 27211 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:14:01.104223 27211 webserver.cc:460] Webserver started at http://127.9.236.1:43323/ using document root <none> and password file <none>
I20260929 19:14:01.104468 27211 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:01.104545 27211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:01.104696 27211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:01.106083 27211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "4d895871228c468fbcb12d0810e4ac31"
format_stamp: "Formatted at 2026-09-29 19:14:01 on dist-test-slave-wftq"
server_key: "e83d8c35736000c4b7a12db4ae9d889e"
server_key_iv: "b984264f14f3362f8bc8c506697ded53"
server_key_version: "encryptionkey@0"
I20260929 19:14:01.106467 27211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "4d895871228c468fbcb12d0810e4ac31"
format_stamp: "Formatted at 2026-09-29 19:14:01 on dist-test-slave-wftq"
server_key: "e83d8c35736000c4b7a12db4ae9d889e"
server_key_iv: "b984264f14f3362f8bc8c506697ded53"
server_key_version: "encryptionkey@0"
I20260929 19:14:01.108036 27211 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:14:01.108904 27226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:01.109153 27211 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:01.109274 27211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "4d895871228c468fbcb12d0810e4ac31"
format_stamp: "Formatted at 2026-09-29 19:14:01 on dist-test-slave-wftq"
server_key: "e83d8c35736000c4b7a12db4ae9d889e"
server_key_iv: "b984264f14f3362f8bc8c506697ded53"
server_key_version: "encryptionkey@0"
I20260929 19:14:01.109376 27211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:01.128432 27211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:01.130760 27211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:01.130903 27211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:01.131126 27211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:14:01.131438 27211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:14:01.131482 27211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:01.131515 27211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:14:01.131536 27211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:01.138358 27211 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:41977
I20260929 19:14:01.138482 27339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:41977 every 8 connection(s)
I20260929 19:14:01.138787 27211 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:14:01.145756 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 27211
I20260929 19:14:01.145850 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:14:01.146028 10160 external_mini_cluster.cc:1428] Setting key c217a61f594a2aee9d8b079e84b7a2b4
Sep 29 19:14:01 dist-test-slave-wftq krb5kdc[27104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709241, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:14:01.150015 27206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.139652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:54201 (local address 127.9.236.62:35523)
0929 19:14:01.139730 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:01.139733 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:01.139935 (+ 202us) server_negotiation.cc:407] Connection header received
0929 19:14:01.140764 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:01.140765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:01.140780 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:01.140825 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:01.142273 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.142797 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.144145 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.144338 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.144395 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:01.147175 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:01.147177 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:01.147177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:01.147186 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:01.148650 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:01.149085 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:01.149086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:01.149086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:01.149171 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:01.149397 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:01.149398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:01.149398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:01.149510 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:01.149564 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:01.149831 (+ 267us) server_negotiation.cc:299] Negotiation successful
0929 19:14:01.149923 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 19:14:01.150198 27342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.139565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:35523 (local address 127.9.236.1:54201)
0929 19:14:01.139743 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:01.139802 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:01.140640 (+ 838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:01.140924 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:01.140929 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:01.141827 (+ 898us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:01.142170 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:01.142177 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.142890 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.142891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:01.144070 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:01.144073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.144456 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.144457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:01.144594 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:01.145178 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:01.145190 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:01.147071 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:01.148752 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:01.148762 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:01.148765 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:01.149015 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:01.149232 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:01.149233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:01.149233 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:01.149340 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:01.149564 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:01.149570 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:01.149713 (+ 143us) client_negotiation.cc:769] Sending connection context
0929 19:14:01.149803 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 19:14:01.149984 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":87,"threads_started":1}
I20260929 19:14:01.150712 27340 heartbeater.cc:347] Connected to a master server at 127.9.236.62:35523
I20260929 19:14:01.150815 27340 heartbeater.cc:464] Registering TS with master...
I20260929 19:14:01.151001 27340 heartbeater.cc:511] Master 127.9.236.62:35523 requested a full tablet report, sending...
I20260929 19:14:01.151592 27149 ts_manager.cc:194] Registered new tserver with Master: 4d895871228c468fbcb12d0810e4ac31 (127.9.236.1:41977)
I20260929 19:14:01.152467 27149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:54201
WARNING: no policy specified for kudu/127.9.236.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
I20260929 19:14:01.171603 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:35523
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:34367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:01.257381 27345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:01.257603 27345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:01.257628 27345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:01.258919 27345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:01.258962 27345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:01.258991 27345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:14:01.260546 27345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:34367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:35523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:01.260790 27345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:01.261031 27345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:14:01.266507 27345 server_base.cc:1034] running on GCE node
W20260929 19:14:01.266476 27350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:01.266489 27351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:01.266485 27353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:01.267035 27345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:01.267352 27345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:01.268496 27345 hybrid_clock.cc:648] HybridClock initialized: now 1790709241268474 us; error 68 us; skew 500 ppm
Sep 29 19:14:01 dist-test-slave-wftq krb5kdc[27104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709241, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:01.271005 27345 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:14:01.271435 27345 webserver.cc:460] Webserver started at http://127.9.236.2:45909/ using document root <none> and password file <none>
I20260929 19:14:01.271651 27345 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:01.271697 27345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:01.271817 27345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:01.272819 27345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "0c7f41de03cb4d2baee7b63fa5122649"
format_stamp: "Formatted at 2026-09-29 19:14:01 on dist-test-slave-wftq"
server_key: "146ffc6b43077a200c99131e5b2a385e"
server_key_iv: "e573adac301cb4977315cf88c5c0e036"
server_key_version: "encryptionkey@0"
I20260929 19:14:01.273170 27345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "0c7f41de03cb4d2baee7b63fa5122649"
format_stamp: "Formatted at 2026-09-29 19:14:01 on dist-test-slave-wftq"
server_key: "146ffc6b43077a200c99131e5b2a385e"
server_key_iv: "e573adac301cb4977315cf88c5c0e036"
server_key_version: "encryptionkey@0"
I20260929 19:14:01.274425 27345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:14:01.275167 27360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:01.275352 27345 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:14:01.275434 27345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "0c7f41de03cb4d2baee7b63fa5122649"
format_stamp: "Formatted at 2026-09-29 19:14:01 on dist-test-slave-wftq"
server_key: "146ffc6b43077a200c99131e5b2a385e"
server_key_iv: "e573adac301cb4977315cf88c5c0e036"
server_key_version: "encryptionkey@0"
I20260929 19:14:01.275496 27345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:01.283535 27345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:01.285264 27345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:01.285386 27345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:01.285583 27345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:14:01.285861 27345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:14:01.285893 27345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:01.285919 27345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:14:01.285935 27345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:01.292093 27345 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:36527
I20260929 19:14:01.292183 27473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:36527 every 8 connection(s)
I20260929 19:14:01.292433 27345 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:14:01.296540 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 27345
I20260929 19:14:01.296626 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:14:01.296801 10160 external_mini_cluster.cc:1428] Setting key 3e45d641692d500a26b3393471001274
Sep 29 19:14:01 dist-test-slave-wftq krb5kdc[27104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709241, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:14:01.302052 27206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.293182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:50143 (local address 127.9.236.62:35523)
0929 19:14:01.293263 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:01.293266 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:01.293454 (+ 188us) server_negotiation.cc:407] Connection header received
0929 19:14:01.294022 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:01.294024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:01.294037 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:01.294114 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:01.295111 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.295596 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.296625 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.296867 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.296948 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:01.299330 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:01.299332 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:01.299332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:01.299340 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:01.300723 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:01.301141 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:01.301142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:01.301143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:01.301205 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:01.301425 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:01.301426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:01.301426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:01.301550 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:01.301607 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:01.301859 (+ 252us) server_negotiation.cc:299] Negotiation successful
0929 19:14:01.301952 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 19:14:01.302246 27476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.293135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:35523 (local address 127.9.236.2:50143)
0929 19:14:01.293317 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:01.293359 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:01.293917 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:01.294180 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:01.294184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:01.294804 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:01.295031 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:01.295036 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.295669 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.295670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:01.296530 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:01.296532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.296960 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.296962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:01.297141 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:01.297684 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:01.297694 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:01.299239 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:01.300799 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:01.300805 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:01.300808 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:01.301051 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:01.301263 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:01.301263 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:01.301264 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:01.301369 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:01.301602 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:01.301609 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:01.301741 (+ 132us) client_negotiation.cc:769] Sending connection context
0929 19:14:01.301831 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 19:14:01.302020 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260929 19:14:01.302707 27474 heartbeater.cc:347] Connected to a master server at 127.9.236.62:35523
I20260929 19:14:01.302810 27474 heartbeater.cc:464] Registering TS with master...
I20260929 19:14:01.302994 27474 heartbeater.cc:511] Master 127.9.236.62:35523 requested a full tablet report, sending...
I20260929 19:14:01.303388 27149 ts_manager.cc:194] Registered new tserver with Master: 0c7f41de03cb4d2baee7b63fa5122649 (127.9.236.2:36527)
I20260929 19:14:01.303793 27149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:50143
WARNING: no policy specified for kudu/127.9.236.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
I20260929 19:14:01.321863 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:35523
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:34367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:01.405382 27479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:01.405577 27479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:01.405599 27479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:01.406730 27479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:01.406775 27479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:01.406800 27479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:14:01.408170 27479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:34367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:35523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:01.408344 27479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:01.408526 27479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:01.413329 27485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:01.413318 27484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:01.413614 27479 server_base.cc:1034] running on GCE node
W20260929 19:14:01.413317 27487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:01.413859 27479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:01.414094 27479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:01.415246 27479 hybrid_clock.cc:648] HybridClock initialized: now 1790709241415227 us; error 35 us; skew 500 ppm
Sep 29 19:14:01 dist-test-slave-wftq krb5kdc[27104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709241, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:01.417901 27479 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:14:01.418330 27479 webserver.cc:460] Webserver started at http://127.9.236.3:33631/ using document root <none> and password file <none>
I20260929 19:14:01.418524 27479 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:01.418569 27479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:01.418680 27479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:01.419744 27479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "40eb27f3bccb4937babe0b45d4a9e2ff"
format_stamp: "Formatted at 2026-09-29 19:14:01 on dist-test-slave-wftq"
server_key: "e732bb1f078968b20cbfcd12706ce6ee"
server_key_iv: "54cc690caa59bf99e4ae5d3e4b8add90"
server_key_version: "encryptionkey@0"
I20260929 19:14:01.420053 27479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "40eb27f3bccb4937babe0b45d4a9e2ff"
format_stamp: "Formatted at 2026-09-29 19:14:01 on dist-test-slave-wftq"
server_key: "e732bb1f078968b20cbfcd12706ce6ee"
server_key_iv: "54cc690caa59bf99e4ae5d3e4b8add90"
server_key_version: "encryptionkey@0"
I20260929 19:14:01.421335 27479 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:14:01.422053 27494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:01.422241 27479 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:14:01.422322 27479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "40eb27f3bccb4937babe0b45d4a9e2ff"
format_stamp: "Formatted at 2026-09-29 19:14:01 on dist-test-slave-wftq"
server_key: "e732bb1f078968b20cbfcd12706ce6ee"
server_key_iv: "54cc690caa59bf99e4ae5d3e4b8add90"
server_key_version: "encryptionkey@0"
I20260929 19:14:01.422380 27479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:01.440009 27479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:01.442178 27479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:01.442310 27479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:01.442512 27479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:14:01.442798 27479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:14:01.442831 27479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:01.442860 27479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:14:01.442886 27479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:01.449175 27479 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:40989
I20260929 19:14:01.449272 27607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:40989 every 8 connection(s)
I20260929 19:14:01.449527 27479 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
Sep 29 19:14:01 dist-test-slave-wftq krb5kdc[27104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709241, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:14:01.456779 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 27479
I20260929 19:14:01.456955 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:14:01.457214 10160 external_mini_cluster.cc:1428] Setting key cd1891352da342982695e7385a46ccc4
I20260929 19:14:01.458793 27206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.450293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:42417 (local address 127.9.236.62:35523)
0929 19:14:01.450373 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:01.450376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:01.450597 (+ 221us) server_negotiation.cc:407] Connection header received
0929 19:14:01.451100 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:01.451101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:01.451114 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:01.451172 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:01.452045 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.452579 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.453583 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.453787 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.453843 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:01.455922 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:01.455923 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:01.455923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:01.455930 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:01.457335 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:01.457791 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:01.457792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:01.457793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:01.457852 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:01.458097 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:01.458098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:01.458099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:01.458240 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:01.458300 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:01.458607 (+ 307us) server_negotiation.cc:299] Negotiation successful
0929 19:14:01.458696 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 19:14:01.459007 27610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.450276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:35523 (local address 127.9.236.3:42417)
0929 19:14:01.450466 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:01.450507 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:01.451005 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:01.451230 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:01.451233 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:01.451773 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:01.451963 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:01.451967 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.452648 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.452649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:01.453505 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:01.453507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.453857 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.453858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:01.453951 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:01.454438 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:01.454445 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:01.455820 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:01.457430 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:01.457437 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:01.457440 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:01.457717 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:01.457916 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:01.457916 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:01.457917 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:01.458036 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:01.458299 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:01.458305 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:01.458470 (+ 165us) client_negotiation.cc:769] Sending connection context
0929 19:14:01.458576 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 19:14:01.458760 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":109,"threads_started":1}
I20260929 19:14:01.459465 27608 heartbeater.cc:347] Connected to a master server at 127.9.236.62:35523
I20260929 19:14:01.459558 27608 heartbeater.cc:464] Registering TS with master...
I20260929 19:14:01.459789 27608 heartbeater.cc:511] Master 127.9.236.62:35523 requested a full tablet report, sending...
I20260929 19:14:01.460196 27149 ts_manager.cc:194] Registered new tserver with Master: 40eb27f3bccb4937babe0b45d4a9e2ff (127.9.236.3:40989)
I20260929 19:14:01.460589 27149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:42417
I20260929 19:14:01.468917 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:14:01.476150 27206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.470349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50382 (local address 127.9.236.62:35523)
0929 19:14:01.470407 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:01.470409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:01.470519 (+ 110us) server_negotiation.cc:407] Connection header received
0929 19:14:01.470581 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:01.470582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:01.470592 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:01.470647 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:01.471290 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.471846 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.472618 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.472812 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.472859 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:01.474025 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:01.474026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:01.474026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:01.474032 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:01.475265 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:01.475538 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:01.475539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:01.475539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:01.475582 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:01.475732 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:01.475733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:01.475733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:01.475861 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:01.475934 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:01.476029 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 19:14:01.476086 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 19:14:01.477653 27149 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:14:01.479176 27149 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:14:01.485999 27619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.481572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:41977 (local address 127.0.0.1:57848)
0929 19:14:01.481866 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:01.481875 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:01.481916 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:01.482256 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:01.482259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:01.482260 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:01.482319 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:01.482320 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.483314 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.483315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:01.484747 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:01.484748 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.485687 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.485688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:01.485768 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:01.485782 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 19:14:01.485865 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 19:14:01.485926 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":54,"thread_start_us":110,"threads_started":1}
I20260929 19:14:01.486049 27624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.481876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57848 (local address 127.9.236.1:41977)
0929 19:14:01.482100 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:01.482105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:01.482115 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:14:01.482152 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:01.482154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:01.482166 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:01.482238 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:01.482448 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.483224 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.484835 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.485607 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.485666 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:01.485715 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:01.485898 (+ 183us) server_negotiation.cc:299] Negotiation successful
0929 19:14:01.485947 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":113,"threads_started":1}
I20260929 19:14:01.486460 27621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.481572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:40989 (local address 127.0.0.1:36972)
0929 19:14:01.481990 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:01.481999 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:01.482035 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:01.482391 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:01.482393 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:01.482395 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:01.483440 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:01.483444 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.484308 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.484309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:01.485317 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:01.485320 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.486241 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.486242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:01.486296 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:01.486300 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:14:01.486349 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 19:14:01.486398 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":984,"thread_start_us":109,"threads_started":1}
I20260929 19:14:01.486755 27623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.481851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36972 (local address 127.9.236.3:40989)
0929 19:14:01.482065 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:01.482073 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:01.482086 (+ 13us) server_negotiation.cc:407] Connection header received
0929 19:14:01.482136 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:01.482139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:01.482158 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:01.482460 (+ 302us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:01.483517 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.484182 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.485410 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.486072 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.486143 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:01.486203 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:01.486613 (+ 410us) server_negotiation.cc:299] Negotiation successful
0929 19:14:01.486663 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":90,"threads_started":1}
I20260929 19:14:01.487061 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.481544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:36527 (local address 127.0.0.1:33864)
0929 19:14:01.481783 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:01.481822 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:01.481888 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:01.482391 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:01.482393 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:01.482397 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:01.482485 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:01.482487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.483416 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.483417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:01.484732 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:01.484735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.486793 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.486794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:01.486862 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:01.486866 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:14:01.486921 (+ 55us) client_negotiation.cc:240] Negotiation successful
0929 19:14:01.486985 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":38,"thread_start_us":44,"threads_started":1}
I20260929 19:14:01.487015 27274 tablet_service.cc:1467] Processing CreateTablet for tablet 6d86147509084b46a58a6ecbfaa5ad46 (DEFAULT_TABLE table=test-table [id=816df6e5acc54a80a5cee8940fe953b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:14:01.487154 27622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.481696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33864 (local address 127.9.236.2:36527)
0929 19:14:01.482137 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:01.482142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:01.482152 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:14:01.482196 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:01.482198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:01.482212 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:01.482289 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:01.482561 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.483318 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.485747 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.486644 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.486704 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:01.486755 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:01.486975 (+ 220us) server_negotiation.cc:299] Negotiation successful
0929 19:14:01.487058 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":96,"threads_started":1}
I20260929 19:14:01.487401 27274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d86147509084b46a58a6ecbfaa5ad46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:01.487839 27408 tablet_service.cc:1467] Processing CreateTablet for tablet 6d86147509084b46a58a6ecbfaa5ad46 (DEFAULT_TABLE table=test-table [id=816df6e5acc54a80a5cee8940fe953b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:14:01.488180 27408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d86147509084b46a58a6ecbfaa5ad46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:01.491264 27625 tablet_bootstrap.cc:492] T 6d86147509084b46a58a6ecbfaa5ad46 P 4d895871228c468fbcb12d0810e4ac31: Bootstrap starting.
I20260929 19:14:01.491660 27626 tablet_bootstrap.cc:492] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649: Bootstrap starting.
I20260929 19:14:01.491856 27542 tablet_service.cc:1467] Processing CreateTablet for tablet 6d86147509084b46a58a6ecbfaa5ad46 (DEFAULT_TABLE table=test-table [id=816df6e5acc54a80a5cee8940fe953b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:14:01.492014 27625 tablet_bootstrap.cc:654] T 6d86147509084b46a58a6ecbfaa5ad46 P 4d895871228c468fbcb12d0810e4ac31: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:01.492219 27542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d86147509084b46a58a6ecbfaa5ad46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:01.492354 27626 tablet_bootstrap.cc:654] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:01.492537 27625 log.cc:824] T 6d86147509084b46a58a6ecbfaa5ad46 P 4d895871228c468fbcb12d0810e4ac31: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:01.492956 27626 log.cc:824] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:01.493471 27625 tablet_bootstrap.cc:492] T 6d86147509084b46a58a6ecbfaa5ad46 P 4d895871228c468fbcb12d0810e4ac31: No bootstrap required, opened a new log
I20260929 19:14:01.493548 27625 ts_tablet_manager.cc:1397] T 6d86147509084b46a58a6ecbfaa5ad46 P 4d895871228c468fbcb12d0810e4ac31: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:14:01.493918 27626 tablet_bootstrap.cc:492] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649: No bootstrap required, opened a new log
I20260929 19:14:01.494015 27626 ts_tablet_manager.cc:1397] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:14:01.495196 27625 raft_consensus.cc:348] T 6d86147509084b46a58a6ecbfaa5ad46 P 4d895871228c468fbcb12d0810e4ac31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7f41de03cb4d2baee7b63fa5122649" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36527 } } peers { permanent_uuid: "40eb27f3bccb4937babe0b45d4a9e2ff" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40989 } } peers { permanent_uuid: "4d895871228c468fbcb12d0810e4ac31" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41977 } }
I20260929 19:14:01.495343 27625 raft_consensus.cc:374] T 6d86147509084b46a58a6ecbfaa5ad46 P 4d895871228c468fbcb12d0810e4ac31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:01.495375 27625 raft_consensus.cc:729] T 6d86147509084b46a58a6ecbfaa5ad46 P 4d895871228c468fbcb12d0810e4ac31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d895871228c468fbcb12d0810e4ac31, State: Initialized, Role: FOLLOWER
I20260929 19:14:01.495501 27625 consensus_queue.cc:260] T 6d86147509084b46a58a6ecbfaa5ad46 P 4d895871228c468fbcb12d0810e4ac31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7f41de03cb4d2baee7b63fa5122649" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36527 } } peers { permanent_uuid: "40eb27f3bccb4937babe0b45d4a9e2ff" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40989 } } peers { permanent_uuid: "4d895871228c468fbcb12d0810e4ac31" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41977 } }
I20260929 19:14:01.495777 27625 ts_tablet_manager.cc:1428] T 6d86147509084b46a58a6ecbfaa5ad46 P 4d895871228c468fbcb12d0810e4ac31: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 19:14:01.495896 27340 heartbeater.cc:503] Master 127.9.236.62:35523 was elected leader, sending a full tablet report...
I20260929 19:14:01.495926 27629 tablet_bootstrap.cc:492] T 6d86147509084b46a58a6ecbfaa5ad46 P 40eb27f3bccb4937babe0b45d4a9e2ff: Bootstrap starting.
I20260929 19:14:01.496129 27626 raft_consensus.cc:348] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7f41de03cb4d2baee7b63fa5122649" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36527 } } peers { permanent_uuid: "40eb27f3bccb4937babe0b45d4a9e2ff" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40989 } } peers { permanent_uuid: "4d895871228c468fbcb12d0810e4ac31" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41977 } }
I20260929 19:14:01.496335 27626 raft_consensus.cc:374] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:01.496377 27626 raft_consensus.cc:729] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c7f41de03cb4d2baee7b63fa5122649, State: Initialized, Role: FOLLOWER
I20260929 19:14:01.496505 27626 consensus_queue.cc:260] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7f41de03cb4d2baee7b63fa5122649" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36527 } } peers { permanent_uuid: "40eb27f3bccb4937babe0b45d4a9e2ff" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40989 } } peers { permanent_uuid: "4d895871228c468fbcb12d0810e4ac31" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41977 } }
I20260929 19:14:01.496618 27629 tablet_bootstrap.cc:654] T 6d86147509084b46a58a6ecbfaa5ad46 P 40eb27f3bccb4937babe0b45d4a9e2ff: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:01.496801 27626 ts_tablet_manager.cc:1428] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:14:01.496924 27474 heartbeater.cc:503] Master 127.9.236.62:35523 was elected leader, sending a full tablet report...
I20260929 19:14:01.497197 27629 log.cc:824] T 6d86147509084b46a58a6ecbfaa5ad46 P 40eb27f3bccb4937babe0b45d4a9e2ff: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:01.498034 27629 tablet_bootstrap.cc:492] T 6d86147509084b46a58a6ecbfaa5ad46 P 40eb27f3bccb4937babe0b45d4a9e2ff: No bootstrap required, opened a new log
I20260929 19:14:01.498106 27629 ts_tablet_manager.cc:1397] T 6d86147509084b46a58a6ecbfaa5ad46 P 40eb27f3bccb4937babe0b45d4a9e2ff: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:14:01.499431 27629 raft_consensus.cc:348] T 6d86147509084b46a58a6ecbfaa5ad46 P 40eb27f3bccb4937babe0b45d4a9e2ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7f41de03cb4d2baee7b63fa5122649" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36527 } } peers { permanent_uuid: "40eb27f3bccb4937babe0b45d4a9e2ff" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40989 } } peers { permanent_uuid: "4d895871228c468fbcb12d0810e4ac31" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41977 } }
I20260929 19:14:01.499567 27629 raft_consensus.cc:374] T 6d86147509084b46a58a6ecbfaa5ad46 P 40eb27f3bccb4937babe0b45d4a9e2ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:01.499586 27629 raft_consensus.cc:729] T 6d86147509084b46a58a6ecbfaa5ad46 P 40eb27f3bccb4937babe0b45d4a9e2ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40eb27f3bccb4937babe0b45d4a9e2ff, State: Initialized, Role: FOLLOWER
I20260929 19:14:01.499699 27629 consensus_queue.cc:260] T 6d86147509084b46a58a6ecbfaa5ad46 P 40eb27f3bccb4937babe0b45d4a9e2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7f41de03cb4d2baee7b63fa5122649" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36527 } } peers { permanent_uuid: "40eb27f3bccb4937babe0b45d4a9e2ff" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40989 } } peers { permanent_uuid: "4d895871228c468fbcb12d0810e4ac31" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41977 } }
I20260929 19:14:01.499948 27629 ts_tablet_manager.cc:1428] T 6d86147509084b46a58a6ecbfaa5ad46 P 40eb27f3bccb4937babe0b45d4a9e2ff: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:14:01.500005 27608 heartbeater.cc:503] Master 127.9.236.62:35523 was elected leader, sending a full tablet report...
W20260929 19:14:01.543020 27475 tablet.cc:2369] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:14:01.550442 27631 raft_consensus.cc:482] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:14:01.550561 27631 raft_consensus.cc:504] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7f41de03cb4d2baee7b63fa5122649" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36527 } } peers { permanent_uuid: "40eb27f3bccb4937babe0b45d4a9e2ff" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40989 } } peers { permanent_uuid: "4d895871228c468fbcb12d0810e4ac31" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41977 } }
I20260929 19:14:01.550951 27631 leader_election.cc:288] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40eb27f3bccb4937babe0b45d4a9e2ff (127.9.236.3:40989), 4d895871228c468fbcb12d0810e4ac31 (127.9.236.1:41977)
I20260929 19:14:01.554351 27624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.551279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:51223 (local address 127.9.236.1:41977)
0929 19:14:01.551338 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:01.551340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:01.551376 (+ 36us) server_negotiation.cc:407] Connection header received
0929 19:14:01.551479 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:01.551480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:01.551494 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:01.551537 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:01.551716 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.552329 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.553412 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.553927 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.553999 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:01.554014 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:01.554224 (+ 210us) server_negotiation.cc:299] Negotiation successful
0929 19:14:01.554270 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 19:14:01.554351 27476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.551202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:41977 (local address 127.9.236.2:51223)
0929 19:14:01.551306 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:01.551322 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:01.551398 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:01.551551 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:01.551551 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:01.551554 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:01.551629 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:01.551631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.552433 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.552434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:01.553332 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:01.553334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.554065 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.554066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:01.554160 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:01.554163 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:14:01.554209 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 19:14:01.554265 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":22}
I20260929 19:14:01.554427 27623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.551374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:35389 (local address 127.9.236.3:40989)
0929 19:14:01.551451 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:01.551453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:01.551599 (+ 146us) server_negotiation.cc:407] Connection header received
0929 19:14:01.551655 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:01.551657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:01.551669 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:01.551735 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:01.551909 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.552405 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.553540 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.554057 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.554120 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:01.554133 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:01.554314 (+ 181us) server_negotiation.cc:299] Negotiation successful
0929 19:14:01.554352 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 19:14:01.554724 27294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d86147509084b46a58a6ecbfaa5ad46" candidate_uuid: "0c7f41de03cb4d2baee7b63fa5122649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d895871228c468fbcb12d0810e4ac31" is_pre_election: true
I20260929 19:14:01.554868 27634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.551202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:40989 (local address 127.9.236.2:35389)
0929 19:14:01.551528 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:01.551538 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:01.551598 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:01.551749 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:01.551751 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:01.551754 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:01.551842 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:01.551844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.552484 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.552484 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:01.553439 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:01.553441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.554173 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.554174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:01.554244 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:01.554250 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 19:14:01.554296 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 19:14:01.554353 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":181,"thread_start_us":139,"threads_started":1}
I20260929 19:14:01.554895 27294 raft_consensus.cc:2454] T 6d86147509084b46a58a6ecbfaa5ad46 P 4d895871228c468fbcb12d0810e4ac31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c7f41de03cb4d2baee7b63fa5122649 in term 0.
I20260929 19:14:01.555123 27361 leader_election.cc:302] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c7f41de03cb4d2baee7b63fa5122649, 4d895871228c468fbcb12d0810e4ac31; no voters:
I20260929 19:14:01.555284 27631 raft_consensus.cc:2781] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:14:01.555289 27562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d86147509084b46a58a6ecbfaa5ad46" candidate_uuid: "0c7f41de03cb4d2baee7b63fa5122649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40eb27f3bccb4937babe0b45d4a9e2ff" is_pre_election: true
I20260929 19:14:01.555343 27631 raft_consensus.cc:482] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:14:01.555373 27631 raft_consensus.cc:3037] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:01.555419 27562 raft_consensus.cc:2454] T 6d86147509084b46a58a6ecbfaa5ad46 P 40eb27f3bccb4937babe0b45d4a9e2ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c7f41de03cb4d2baee7b63fa5122649 in term 0.
I20260929 19:14:01.556170 27631 raft_consensus.cc:504] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7f41de03cb4d2baee7b63fa5122649" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36527 } } peers { permanent_uuid: "40eb27f3bccb4937babe0b45d4a9e2ff" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40989 } } peers { permanent_uuid: "4d895871228c468fbcb12d0810e4ac31" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41977 } }
I20260929 19:14:01.556301 27631 leader_election.cc:288] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 [CANDIDATE]: Term 1 election: Requested vote from peers 40eb27f3bccb4937babe0b45d4a9e2ff (127.9.236.3:40989), 4d895871228c468fbcb12d0810e4ac31 (127.9.236.1:41977)
I20260929 19:14:01.556519 27294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d86147509084b46a58a6ecbfaa5ad46" candidate_uuid: "0c7f41de03cb4d2baee7b63fa5122649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d895871228c468fbcb12d0810e4ac31"
I20260929 19:14:01.556538 27562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d86147509084b46a58a6ecbfaa5ad46" candidate_uuid: "0c7f41de03cb4d2baee7b63fa5122649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40eb27f3bccb4937babe0b45d4a9e2ff"
I20260929 19:14:01.556602 27562 raft_consensus.cc:3037] T 6d86147509084b46a58a6ecbfaa5ad46 P 40eb27f3bccb4937babe0b45d4a9e2ff [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:01.556604 27294 raft_consensus.cc:3037] T 6d86147509084b46a58a6ecbfaa5ad46 P 4d895871228c468fbcb12d0810e4ac31 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:01.557488 27562 raft_consensus.cc:2454] T 6d86147509084b46a58a6ecbfaa5ad46 P 40eb27f3bccb4937babe0b45d4a9e2ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c7f41de03cb4d2baee7b63fa5122649 in term 1.
I20260929 19:14:01.557488 27294 raft_consensus.cc:2454] T 6d86147509084b46a58a6ecbfaa5ad46 P 4d895871228c468fbcb12d0810e4ac31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c7f41de03cb4d2baee7b63fa5122649 in term 1.
I20260929 19:14:01.557685 27362 leader_election.cc:302] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c7f41de03cb4d2baee7b63fa5122649, 40eb27f3bccb4937babe0b45d4a9e2ff; no voters:
I20260929 19:14:01.557802 27631 raft_consensus.cc:2781] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:01.557962 27631 raft_consensus.cc:686] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 [term 1 LEADER]: Becoming Leader. State: Replica: 0c7f41de03cb4d2baee7b63fa5122649, State: Running, Role: LEADER
I20260929 19:14:01.558048 27631 consensus_queue.cc:237] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7f41de03cb4d2baee7b63fa5122649" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36527 } } peers { permanent_uuid: "40eb27f3bccb4937babe0b45d4a9e2ff" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40989 } } peers { permanent_uuid: "4d895871228c468fbcb12d0810e4ac31" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41977 } }
I20260929 19:14:01.558974 27149 catalog_manager.cc:5495] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c7f41de03cb4d2baee7b63fa5122649 (127.9.236.2). New cstate: current_term: 1 leader_uuid: "0c7f41de03cb4d2baee7b63fa5122649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7f41de03cb4d2baee7b63fa5122649" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40eb27f3bccb4937babe0b45d4a9e2ff" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d895871228c468fbcb12d0810e4ac31" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41977 } health_report { overall_health: UNKNOWN } } }
I20260929 19:14:01.576821 27622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.574634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33866 (local address 127.9.236.2:36527)
0929 19:14:01.574678 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:01.574680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:01.574710 (+ 30us) server_negotiation.cc:407] Connection header received
0929 19:14:01.574742 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:01.574743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:01.574762 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:01.574819 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:01.574994 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.575459 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.576237 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.576416 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.576470 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:01.576573 (+ 103us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:01.576672 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:01.576734 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:01.576750 (+ 16us) server_negotiation.cc:299] Negotiation successful
0929 19:14:01.576765 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:14:01.578770 27294 raft_consensus.cc:1261] T 6d86147509084b46a58a6ecbfaa5ad46 P 4d895871228c468fbcb12d0810e4ac31 [term 1 FOLLOWER]: Refusing update from remote peer 0c7f41de03cb4d2baee7b63fa5122649: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:14:01.578821 27562 raft_consensus.cc:1261] T 6d86147509084b46a58a6ecbfaa5ad46 P 40eb27f3bccb4937babe0b45d4a9e2ff [term 1 FOLLOWER]: Refusing update from remote peer 0c7f41de03cb4d2baee7b63fa5122649: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:14:01.579133 27631 consensus_queue.cc:1035] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d895871228c468fbcb12d0810e4ac31" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 41977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:14:01.579362 27635 consensus_queue.cc:1035] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40eb27f3bccb4937babe0b45d4a9e2ff" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:14:01.582242 27639 mvcc.cc:205] Tried to move back new op lower bound from 7334745053503791104 to 7334745053422338048. Current Snapshot: MvccSnapshot[applied={T|T < 7334745053503791104}]
I20260929 19:14:01.582917 27641 mvcc.cc:205] Tried to move back new op lower bound from 7334745053503791104 to 7334745053422338048. Current Snapshot: MvccSnapshot[applied={T|T < 7334745053503791104 or (T in {7334745053503791104})}]
I20260929 19:14:01.584350 27148 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:14:01.584476 27148 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:14:01.585366 27148 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bcba2cfc1ddc4357acb3c721f53cca90: Sending DeleteTablet for 3 replicas of tablet 6d86147509084b46a58a6ecbfaa5ad46
I20260929 19:14:01.585685 27408 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d86147509084b46a58a6ecbfaa5ad46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:14:01 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:33864
I20260929 19:14:01.585692 27542 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d86147509084b46a58a6ecbfaa5ad46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:14:01 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:36972
I20260929 19:14:01.585816 27274 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d86147509084b46a58a6ecbfaa5ad46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:14:01 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:57848
I20260929 19:14:01.585968 27647 tablet_replica.cc:330] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649: stopping tablet replica
I20260929 19:14:01.586084 27649 tablet_replica.cc:330] T 6d86147509084b46a58a6ecbfaa5ad46 P 4d895871228c468fbcb12d0810e4ac31: stopping tablet replica
I20260929 19:14:01.586092 27647 raft_consensus.cc:2229] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:14:01.586176 27649 raft_consensus.cc:2229] T 6d86147509084b46a58a6ecbfaa5ad46 P 4d895871228c468fbcb12d0810e4ac31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:14:01.586198 27647 raft_consensus.cc:2258] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:14:01.586270 27649 raft_consensus.cc:2258] T 6d86147509084b46a58a6ecbfaa5ad46 P 4d895871228c468fbcb12d0810e4ac31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:14:01.586395 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 27211
I20260929 19:14:01.586484 27648 tablet_replica.cc:330] T 6d86147509084b46a58a6ecbfaa5ad46 P 40eb27f3bccb4937babe0b45d4a9e2ff: stopping tablet replica
I20260929 19:14:01.586572 27648 raft_consensus.cc:2229] T 6d86147509084b46a58a6ecbfaa5ad46 P 40eb27f3bccb4937babe0b45d4a9e2ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:14:01.586658 27648 raft_consensus.cc:2258] T 6d86147509084b46a58a6ecbfaa5ad46 P 40eb27f3bccb4937babe0b45d4a9e2ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:14:01.587271 27647 ts_tablet_manager.cc:1905] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:14:01.588387 27648 ts_tablet_manager.cc:1905] T 6d86147509084b46a58a6ecbfaa5ad46 P 40eb27f3bccb4937babe0b45d4a9e2ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:14:01.589237 27647 ts_tablet_manager.cc:1918] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:14:01.589308 27647 log.cc:1196] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/6d86147509084b46a58a6ecbfaa5ad46
I20260929 19:14:01.589663 27647 ts_tablet_manager.cc:1939] T 6d86147509084b46a58a6ecbfaa5ad46 P 0c7f41de03cb4d2baee7b63fa5122649: Deleting consensus metadata
I20260929 19:14:01.590054 27135 catalog_manager.cc:4841] TS 0c7f41de03cb4d2baee7b63fa5122649 (127.9.236.2:36527): tablet 6d86147509084b46a58a6ecbfaa5ad46 (table test-table [id=816df6e5acc54a80a5cee8940fe953b2]) successfully deleted
I20260929 19:14:01.590215 27648 ts_tablet_manager.cc:1918] T 6d86147509084b46a58a6ecbfaa5ad46 P 40eb27f3bccb4937babe0b45d4a9e2ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:14:01.590277 27648 log.cc:1196] T 6d86147509084b46a58a6ecbfaa5ad46 P 40eb27f3bccb4937babe0b45d4a9e2ff: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/6d86147509084b46a58a6ecbfaa5ad46
I20260929 19:14:01.590547 27648 ts_tablet_manager.cc:1939] T 6d86147509084b46a58a6ecbfaa5ad46 P 40eb27f3bccb4937babe0b45d4a9e2ff: Deleting consensus metadata
I20260929 19:14:01.591001 27134 catalog_manager.cc:4841] TS 40eb27f3bccb4937babe0b45d4a9e2ff (127.9.236.3:40989): tablet 6d86147509084b46a58a6ecbfaa5ad46 (table test-table [id=816df6e5acc54a80a5cee8940fe953b2]) successfully deleted
W20260929 19:14:01.591562 27133 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.236.1:41977 (error 108)
I20260929 19:14:01.592118 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.591963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:41977 (local address 127.0.0.1:57850)
0929 19:14:01.592028 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:01.592078 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.1:41977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260929 19:14:01.592147 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 27345
W20260929 19:14:01.592828 27133 catalog_manager.cc:4584] TS 4d895871228c468fbcb12d0810e4ac31 (127.9.236.1:41977): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6d86147509084b46a58a6ecbfaa5ad46: Network error: Client connection negotiation failed: client connection to 127.9.236.1:41977: connect: Connection refused (error 111)
I20260929 19:14:01.596596 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 27479
I20260929 19:14:01.601257 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 27117
2026-09-29T19:14:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1863 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-29T19:14:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:01Z Disabled control of system clock
I20260929 19:14:01.629973 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:45503
--webserver_interface=127.9.236.62
--webserver_port=0
--builtin_ntp_servers=127.9.236.20:42257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:45503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 19:14:01.717764 27654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:01.718016 27654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:01.718036 27654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:01.719241 27654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:01.719287 27654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:01.719300 27654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:01.719312 27654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:01.719323 27654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:01.720723 27654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:42257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:45503
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:45503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:01.720955 27654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:01.721300 27654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:01.726374 27660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:01.726338 27662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:01.726367 27659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:01.726513 27654 server_base.cc:1034] running on GCE node
I20260929 19:14:01.726902 27654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:01.727188 27654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:01.728329 27654 hybrid_clock.cc:648] HybridClock initialized: now 1790709241728302 us; error 41 us; skew 500 ppm
I20260929 19:14:01.729629 27654 webserver.cc:460] Webserver started at http://127.9.236.62:36633/ using document root <none> and password file <none>
I20260929 19:14:01.729858 27654 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:01.729907 27654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:01.730044 27654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:01.731168 27654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "42cdb2aa57444a4585a56703ba10cbc3"
format_stamp: "Formatted at 2026-09-29 19:14:01 on dist-test-slave-wftq"
server_key: "3887ad13607636579eaea895de58f13c"
server_key_iv: "64fc5d6421f1256ee18c1474ed0f20d3"
server_key_version: "encryptionkey@0"
I20260929 19:14:01.731515 27654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "42cdb2aa57444a4585a56703ba10cbc3"
format_stamp: "Formatted at 2026-09-29 19:14:01 on dist-test-slave-wftq"
server_key: "3887ad13607636579eaea895de58f13c"
server_key_iv: "64fc5d6421f1256ee18c1474ed0f20d3"
server_key_version: "encryptionkey@0"
I20260929 19:14:01.732775 27654 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:14:01.733546 27668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:01.733701 27654 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:01.733778 27654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "42cdb2aa57444a4585a56703ba10cbc3"
format_stamp: "Formatted at 2026-09-29 19:14:01 on dist-test-slave-wftq"
server_key: "3887ad13607636579eaea895de58f13c"
server_key_iv: "64fc5d6421f1256ee18c1474ed0f20d3"
server_key_version: "encryptionkey@0"
I20260929 19:14:01.733852 27654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:01.750298 27654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:01.750665 27654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:01.750803 27654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:01.755522 27654 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:45503
I20260929 19:14:01.755596 27720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:45503 every 8 connection(s)
I20260929 19:14:01.755936 27654 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:01.757012 27721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:01.759327 27721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3: Bootstrap starting.
I20260929 19:14:01.759864 27721 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:01.760094 27721 log.cc:824] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:01.760808 27721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3: No bootstrap required, opened a new log
I20260929 19:14:01.762337 27721 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42cdb2aa57444a4585a56703ba10cbc3" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 45503 } }
I20260929 19:14:01.762475 27721 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:01.762494 27721 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42cdb2aa57444a4585a56703ba10cbc3, State: Initialized, Role: FOLLOWER
I20260929 19:14:01.762598 27721 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42cdb2aa57444a4585a56703ba10cbc3" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 45503 } }
I20260929 19:14:01.762651 27721 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:01.762679 27721 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:01.762715 27721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:01.763175 27721 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42cdb2aa57444a4585a56703ba10cbc3" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 45503 } }
I20260929 19:14:01.763269 27721 leader_election.cc:302] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42cdb2aa57444a4585a56703ba10cbc3; no voters:
I20260929 19:14:01.763420 27721 leader_election.cc:288] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:01.763516 27724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:01.763734 27721 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:01.763717 27724 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3 [term 1 LEADER]: Becoming Leader. State: Replica: 42cdb2aa57444a4585a56703ba10cbc3, State: Running, Role: LEADER
I20260929 19:14:01.763922 27724 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42cdb2aa57444a4585a56703ba10cbc3" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 45503 } }
I20260929 19:14:01.764410 27726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42cdb2aa57444a4585a56703ba10cbc3. Latest consensus state: current_term: 1 leader_uuid: "42cdb2aa57444a4585a56703ba10cbc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42cdb2aa57444a4585a56703ba10cbc3" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 45503 } } }
I20260929 19:14:01.764513 27726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:01.764492 27725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42cdb2aa57444a4585a56703ba10cbc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42cdb2aa57444a4585a56703ba10cbc3" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 45503 } } }
I20260929 19:14:01.764570 27725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:01.765070 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 27654
I20260929 19:14:01.765204 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:01.765410 10160 external_mini_cluster.cc:1428] Setting key 12ad87394a5c1c7db48482bff472db16
I20260929 19:14:01.764787 27733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:01.766187 27733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:01.767695 27733 catalog_manager.cc:1347] Generated new cluster ID: 349d2b1cbbcf4f9bb9025b418a893b24
I20260929 19:14:01.767745 27733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:01.774724 27743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.765931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47228 (local address 127.9.236.62:45503)
0929 19:14:01.766162 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:01.766172 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:01.769296 (+ 3124us) server_negotiation.cc:407] Connection header received
0929 19:14:01.769452 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:01.769459 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:01.769529 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:01.769667 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:01.769961 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.771366 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.772476 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.773141 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.773280 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:01.774005 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:01.774008 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:01.774009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:01.774225 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:01.774268 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:01.774283 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:01.774357 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:01.774435 (+ 78us) server_negotiation.cc:299] Negotiation successful
0929 19:14:01.774513 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":110,"threads_started":1}
I20260929 19:14:01.778568 27733 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:01.779047 27733 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:01.783780 27733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3: Generated new TSK 0
I20260929 19:14:01.783953 27733 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:01.789007 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:45503
--builtin_ntp_servers=127.9.236.20:42257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 19:14:01.877094 27745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:01.877393 27745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:01.877417 27745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:01.878635 27745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:01.878681 27745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:01.878708 27745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:14:01.880106 27745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:42257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:45503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:01.880353 27745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:01.880681 27745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:01.886058 27750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:01.886153 27751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:01.886199 27753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:01.886368 27745 server_base.cc:1034] running on GCE node
I20260929 19:14:01.886576 27745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:01.886819 27745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:01.887986 27745 hybrid_clock.cc:648] HybridClock initialized: now 1790709241887976 us; error 29 us; skew 500 ppm
I20260929 19:14:01.889374 27745 webserver.cc:460] Webserver started at http://127.9.236.1:33661/ using document root <none> and password file <none>
I20260929 19:14:01.889611 27745 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:01.889662 27745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:01.889792 27745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:01.890925 27745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "827d104c02dc4b4ea9e3e619fbc20432"
format_stamp: "Formatted at 2026-09-29 19:14:01 on dist-test-slave-wftq"
server_key: "463225a61d37b240c31712bd54530120"
server_key_iv: "dcb5d23c5d428252b69d58f55a8cab61"
server_key_version: "encryptionkey@0"
I20260929 19:14:01.891254 27745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "827d104c02dc4b4ea9e3e619fbc20432"
format_stamp: "Formatted at 2026-09-29 19:14:01 on dist-test-slave-wftq"
server_key: "463225a61d37b240c31712bd54530120"
server_key_iv: "dcb5d23c5d428252b69d58f55a8cab61"
server_key_version: "encryptionkey@0"
I20260929 19:14:01.892582 27745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:14:01.893448 27759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:01.893654 27745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:01.893742 27745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "827d104c02dc4b4ea9e3e619fbc20432"
format_stamp: "Formatted at 2026-09-29 19:14:01 on dist-test-slave-wftq"
server_key: "463225a61d37b240c31712bd54530120"
server_key_iv: "dcb5d23c5d428252b69d58f55a8cab61"
server_key_version: "encryptionkey@0"
I20260929 19:14:01.893803 27745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:01.909586 27745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:01.909961 27745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:01.910099 27745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:01.910362 27745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:14:01.910719 27745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:14:01.910753 27745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:01.910785 27745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:14:01.910808 27745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:01.917953 27745 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:46167
I20260929 19:14:01.918076 27872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:46167 every 8 connection(s)
I20260929 19:14:01.918378 27745 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:14:01.924186 27743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.919204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:50739 (local address 127.9.236.62:45503)
0929 19:14:01.919284 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:01.919286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:01.919513 (+ 227us) server_negotiation.cc:407] Connection header received
0929 19:14:01.920092 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:01.920094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:01.920111 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:01.920204 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:01.921009 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.921663 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.922745 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.922963 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.923023 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:01.923909 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:01.923910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:01.923910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:01.923915 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:01.923931 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:01.923936 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:01.923983 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:01.924067 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 19:14:01.924101 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 19:14:01.924399 27875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:01.919162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:45503 (local address 127.9.236.1:50739)
0929 19:14:01.919383 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:01.919426 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:01.919950 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:01.920329 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:01.920333 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:01.920342 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:01.920899 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:01.920904 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.921769 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.921771 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:01.922613 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:01.922616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:01.923039 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:01.923040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:01.923187 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:01.923768 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:14:01.923769 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:01.923797 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:14:01.923819 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:01.923991 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:01.923996 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:01.924000 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:14:01.924061 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 19:14:01.924177 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":109,"threads_started":1}
I20260929 19:14:01.924574 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 27745
I20260929 19:14:01.924681 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:14:01.924877 10160 external_mini_cluster.cc:1428] Setting key 6c180f8c371d986ae93d38977e792b0a
I20260929 19:14:01.925381 27873 heartbeater.cc:347] Connected to a master server at 127.9.236.62:45503
I20260929 19:14:01.925488 27873 heartbeater.cc:464] Registering TS with master...
I20260929 19:14:01.925685 27873 heartbeater.cc:511] Master 127.9.236.62:45503 requested a full tablet report, sending...
I20260929 19:14:01.926188 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:45503
--builtin_ntp_servers=127.9.236.20:42257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 19:14:01.926293 27684 ts_manager.cc:194] Registered new tserver with Master: 827d104c02dc4b4ea9e3e619fbc20432 (127.9.236.1:46167)
I20260929 19:14:01.926932 27684 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.236.1:50739
W20260929 19:14:02.011564 27876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:02.011794 27876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:02.011816 27876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:02.013012 27876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:02.013056 27876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:02.013080 27876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:14:02.014431 27876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:42257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:45503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:02.014659 27876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:02.014969 27876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:02.020340 27884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:02.020318 27882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:02.020335 27881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:02.020380 27876 server_base.cc:1034] running on GCE node
I20260929 19:14:02.020816 27876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:02.021078 27876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:02.022267 27876 hybrid_clock.cc:648] HybridClock initialized: now 1790709242022222 us; error 61 us; skew 500 ppm
I20260929 19:14:02.023638 27876 webserver.cc:460] Webserver started at http://127.9.236.2:34437/ using document root <none> and password file <none>
I20260929 19:14:02.023849 27876 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:02.023891 27876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:02.024003 27876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:02.025194 27876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "108b80712447401e8c55bfe2ca9069b7"
format_stamp: "Formatted at 2026-09-29 19:14:02 on dist-test-slave-wftq"
server_key: "c387755894b7b382dc4c511b0981cdf0"
server_key_iv: "e333737da04c3de1eaaea38788292ce3"
server_key_version: "encryptionkey@0"
I20260929 19:14:02.025539 27876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "108b80712447401e8c55bfe2ca9069b7"
format_stamp: "Formatted at 2026-09-29 19:14:02 on dist-test-slave-wftq"
server_key: "c387755894b7b382dc4c511b0981cdf0"
server_key_iv: "e333737da04c3de1eaaea38788292ce3"
server_key_version: "encryptionkey@0"
I20260929 19:14:02.026971 27876 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:14:02.027787 27890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:02.028035 27876 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:14:02.028129 27876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "108b80712447401e8c55bfe2ca9069b7"
format_stamp: "Formatted at 2026-09-29 19:14:02 on dist-test-slave-wftq"
server_key: "c387755894b7b382dc4c511b0981cdf0"
server_key_iv: "e333737da04c3de1eaaea38788292ce3"
server_key_version: "encryptionkey@0"
I20260929 19:14:02.028213 27876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:02.049345 27876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:02.049727 27876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:02.049881 27876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:02.050155 27876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:14:02.050549 27876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:14:02.050591 27876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:02.050619 27876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:14:02.050648 27876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:02.058058 27876 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:36733
I20260929 19:14:02.058120 28003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:36733 every 8 connection(s)
I20260929 19:14:02.058501 27876 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:14:02.061352 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 27876
I20260929 19:14:02.061468 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:14:02.061666 10160 external_mini_cluster.cc:1428] Setting key e9ad5f72be9d99a8f6667b3123abe7da
I20260929 19:14:02.062733 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:45503
--builtin_ntp_servers=127.9.236.20:42257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 19:14:02.064496 27743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:02.059389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:51019 (local address 127.9.236.62:45503)
0929 19:14:02.059452 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:02.059454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:02.059780 (+ 326us) server_negotiation.cc:407] Connection header received
0929 19:14:02.060325 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:02.060326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:02.060338 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:02.060390 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:02.061117 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.061838 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.062966 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.063185 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.063244 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:02.064130 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:02.064131 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:02.064132 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:02.064136 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:02.064152 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:02.064157 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:02.064186 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:02.064357 (+ 171us) server_negotiation.cc:299] Negotiation successful
0929 19:14:02.064405 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 19:14:02.064828 28006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:02.059362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:45503 (local address 127.9.236.2:51019)
0929 19:14:02.059594 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:02.059633 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:02.060221 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:02.060458 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:02.060463 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:02.060471 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:02.061008 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:02.061012 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.061931 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.061932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:02.062830 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:02.062834 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.063254 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.063255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:02.063383 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:02.063970 (+ 587us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:14:02.063971 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:02.064001 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:14:02.064028 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:02.064227 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:02.064232 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:02.064262 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 19:14:02.064330 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 19:14:02.064439 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":91,"threads_started":1}
I20260929 19:14:02.065486 28004 heartbeater.cc:347] Connected to a master server at 127.9.236.62:45503
I20260929 19:14:02.065593 28004 heartbeater.cc:464] Registering TS with master...
I20260929 19:14:02.065796 28004 heartbeater.cc:511] Master 127.9.236.62:45503 requested a full tablet report, sending...
I20260929 19:14:02.066192 27684 ts_manager.cc:194] Registered new tserver with Master: 108b80712447401e8c55bfe2ca9069b7 (127.9.236.2:36733)
I20260929 19:14:02.066610 27684 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.236.2:51019
W20260929 19:14:02.146960 28007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:02.147163 28007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:02.147183 28007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:02.148321 28007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:02.148365 28007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:02.148388 28007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:14:02.149688 28007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:42257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:45503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:02.149878 28007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:02.150132 28007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:02.155222 28015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:02.155315 28007 server_base.cc:1034] running on GCE node
W20260929 19:14:02.155223 28012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:02.155223 28013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:02.155761 28007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:02.156054 28007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:02.157179 28007 hybrid_clock.cc:648] HybridClock initialized: now 1790709242157156 us; error 42 us; skew 500 ppm
I20260929 19:14:02.158641 28007 webserver.cc:460] Webserver started at http://127.9.236.3:40503/ using document root <none> and password file <none>
I20260929 19:14:02.158959 28007 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:02.159021 28007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:02.159163 28007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:02.160403 28007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "4a6da50914eb49338a1213cdf0388b3e"
format_stamp: "Formatted at 2026-09-29 19:14:02 on dist-test-slave-wftq"
server_key: "ab6663e23a7d2273c09c0c0b3ff5ef2e"
server_key_iv: "300d0419e2953d73e86e1ba2406f3207"
server_key_version: "encryptionkey@0"
I20260929 19:14:02.160781 28007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "4a6da50914eb49338a1213cdf0388b3e"
format_stamp: "Formatted at 2026-09-29 19:14:02 on dist-test-slave-wftq"
server_key: "ab6663e23a7d2273c09c0c0b3ff5ef2e"
server_key_iv: "300d0419e2953d73e86e1ba2406f3207"
server_key_version: "encryptionkey@0"
I20260929 19:14:02.162303 28007 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:14:02.163180 28021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:02.163404 28007 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:14:02.163486 28007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "4a6da50914eb49338a1213cdf0388b3e"
format_stamp: "Formatted at 2026-09-29 19:14:02 on dist-test-slave-wftq"
server_key: "ab6663e23a7d2273c09c0c0b3ff5ef2e"
server_key_iv: "300d0419e2953d73e86e1ba2406f3207"
server_key_version: "encryptionkey@0"
I20260929 19:14:02.163550 28007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:02.179450 28007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:02.179780 28007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:02.179911 28007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:02.180154 28007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:14:02.180495 28007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:14:02.180529 28007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:02.180560 28007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:14:02.180577 28007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:02.187456 28007 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:35875
I20260929 19:14:02.187518 28134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:35875 every 8 connection(s)
I20260929 19:14:02.187876 28007 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:14:02.188493 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 28007
I20260929 19:14:02.188611 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:14:02.188855 10160 external_mini_cluster.cc:1428] Setting key 814c49c810570859eab6262115dfc504
I20260929 19:14:02.193811 27743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:02.188725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:32943 (local address 127.9.236.62:45503)
0929 19:14:02.188782 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:02.188784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:02.189230 (+ 446us) server_negotiation.cc:407] Connection header received
0929 19:14:02.190028 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:02.190029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:02.190041 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:02.190081 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:02.190846 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.191341 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.192401 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.192585 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.192634 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:02.193535 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:02.193536 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:02.193536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:02.193541 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:02.193557 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:02.193561 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:02.193625 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:02.193699 (+ 74us) server_negotiation.cc:299] Negotiation successful
0929 19:14:02.193725 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 19:14:02.193992 28137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:02.188711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:45503 (local address 127.9.236.3:32943)
0929 19:14:02.189001 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:02.189052 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:02.189943 (+ 891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:02.190179 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:02.190183 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:02.190192 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:02.190762 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:02.190767 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.191431 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.191432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:02.192329 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:02.192332 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.192651 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.192652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:02.192765 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:02.193359 (+ 594us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:14:02.193360 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:02.193391 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:14:02.193417 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:02.193632 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:02.193635 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:02.193637 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:14:02.193690 (+ 53us) client_negotiation.cc:240] Negotiation successful
0929 19:14:02.193779 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":146,"threads_started":1}
I20260929 19:14:02.194593 28135 heartbeater.cc:347] Connected to a master server at 127.9.236.62:45503
I20260929 19:14:02.194703 28135 heartbeater.cc:464] Registering TS with master...
I20260929 19:14:02.194895 28135 heartbeater.cc:511] Master 127.9.236.62:45503 requested a full tablet report, sending...
I20260929 19:14:02.195288 27684 ts_manager.cc:194] Registered new tserver with Master: 4a6da50914eb49338a1213cdf0388b3e (127.9.236.3:35875)
I20260929 19:14:02.195662 27684 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.236.3:32943
I20260929 19:14:02.200647 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:14:02.205152 27743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:02.202240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47232 (local address 127.9.236.62:45503)
0929 19:14:02.202323 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:02.202325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:02.202447 (+ 122us) server_negotiation.cc:407] Connection header received
0929 19:14:02.202498 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:02.202499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:02.202512 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:02.202564 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:02.202727 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.203195 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.203983 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.204156 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.204206 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:02.204893 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:02.204894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:02.204894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:02.204898 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:02.204911 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:02.204915 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:02.204956 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:02.205038 (+ 82us) server_negotiation.cc:299] Negotiation successful
0929 19:14:02.205059 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 19:14:02.206816 27684 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:14:02.208336 27684 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:14:02.215310 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:02.210831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37754 (local address 127.9.236.1:46167)
0929 19:14:02.211143 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:02.211148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:02.211157 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:14:02.211195 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:02.211198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:02.211215 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:02.211514 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:02.211589 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.212747 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.214165 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.214873 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.214951 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:02.215050 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:02.215163 (+ 113us) server_negotiation.cc:299] Negotiation successful
0929 19:14:02.215209 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":93,"threads_started":1}
I20260929 19:14:02.215319 28143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:02.210706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:46167 (local address 127.0.0.1:37754)
0929 19:14:02.210850 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:02.210898 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:02.210985 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:02.211335 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:02.211339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:02.211343 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:02.211430 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:02.211433 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.212836 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.212837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:02.214092 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:02.214093 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.214953 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.214954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:02.215049 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:02.215067 (+ 18us) client_negotiation.cc:769] Sending connection context
0929 19:14:02.215147 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 19:14:02.215225 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":82,"thread_start_us":89,"threads_started":1}
I20260929 19:14:02.215319 28146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:02.210710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:35875 (local address 127.0.0.1:41380)
0929 19:14:02.211294 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:02.211306 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:02.211363 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:02.211943 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:02.211944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:02.211947 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:02.211978 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:02.211979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.213143 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.213144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:02.214042 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:02.214044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.214954 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.214955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:02.215049 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:02.215067 (+ 18us) client_negotiation.cc:769] Sending connection context
0929 19:14:02.215142 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 19:14:02.215232 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":12,"thread_start_us":111,"threads_started":1}
I20260929 19:14:02.215616 28147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:02.211095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41380 (local address 127.9.236.3:35875)
0929 19:14:02.211283 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:02.211287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:02.211793 (+ 506us) server_negotiation.cc:407] Connection header received
0929 19:14:02.211834 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:02.211836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:02.211851 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:02.211934 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:02.212040 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.213048 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.214117 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.214867 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.214946 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:02.215053 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:02.215457 (+ 404us) server_negotiation.cc:299] Negotiation successful
0929 19:14:02.215508 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
I20260929 19:14:02.216394 28144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:02.210945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:36733 (local address 127.0.0.1:34672)
0929 19:14:02.211295 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:02.211306 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:02.211363 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:02.211736 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:02.211737 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:02.211739 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:02.211790 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:02.211791 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.212710 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.212711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:02.214089 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:02.214091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.216168 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.216169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:02.216229 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:02.216233 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:14:02.216293 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 19:14:02.216332 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":3,"thread_start_us":68,"threads_started":1}
I20260929 19:14:02.216357 28069 tablet_service.cc:1467] Processing CreateTablet for tablet 89ef358e7b714a43898a8fb44ff3dca5 (DEFAULT_TABLE table=test-table [id=4bef8efe1b6942d0b55c7ec2f7b17db9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:14:02.216393 27807 tablet_service.cc:1467] Processing CreateTablet for tablet 89ef358e7b714a43898a8fb44ff3dca5 (DEFAULT_TABLE table=test-table [id=4bef8efe1b6942d0b55c7ec2f7b17db9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:14:02.216434 28148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:02.211089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34672 (local address 127.9.236.2:36733)
0929 19:14:02.211564 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:02.211568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:02.211578 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:14:02.211622 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:02.211625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:02.211641 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:02.211725 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:02.211871 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.212634 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.215094 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.216082 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.216161 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:02.216240 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:02.216306 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 19:14:02.216349 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":96,"threads_started":1}
I20260929 19:14:02.216790 28069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89ef358e7b714a43898a8fb44ff3dca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:02.216802 27807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89ef358e7b714a43898a8fb44ff3dca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:02.217326 27938 tablet_service.cc:1467] Processing CreateTablet for tablet 89ef358e7b714a43898a8fb44ff3dca5 (DEFAULT_TABLE table=test-table [id=4bef8efe1b6942d0b55c7ec2f7b17db9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:14:02.217657 27938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89ef358e7b714a43898a8fb44ff3dca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:02.219632 28149 tablet_bootstrap.cc:492] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432: Bootstrap starting.
I20260929 19:14:02.220238 28149 tablet_bootstrap.cc:654] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:02.220510 28149 log.cc:824] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:02.220620 28150 tablet_bootstrap.cc:492] T 89ef358e7b714a43898a8fb44ff3dca5 P 4a6da50914eb49338a1213cdf0388b3e: Bootstrap starting.
I20260929 19:14:02.220952 28152 tablet_bootstrap.cc:492] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7: Bootstrap starting.
I20260929 19:14:02.221324 28150 tablet_bootstrap.cc:654] T 89ef358e7b714a43898a8fb44ff3dca5 P 4a6da50914eb49338a1213cdf0388b3e: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:02.221675 28150 log.cc:824] T 89ef358e7b714a43898a8fb44ff3dca5 P 4a6da50914eb49338a1213cdf0388b3e: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:02.221768 28152 tablet_bootstrap.cc:654] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:02.221948 28149 tablet_bootstrap.cc:492] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432: No bootstrap required, opened a new log
I20260929 19:14:02.222038 28149 ts_tablet_manager.cc:1397] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:14:02.222093 28152 log.cc:824] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:02.222604 28150 tablet_bootstrap.cc:492] T 89ef358e7b714a43898a8fb44ff3dca5 P 4a6da50914eb49338a1213cdf0388b3e: No bootstrap required, opened a new log
I20260929 19:14:02.222671 28150 ts_tablet_manager.cc:1397] T 89ef358e7b714a43898a8fb44ff3dca5 P 4a6da50914eb49338a1213cdf0388b3e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:14:02.222942 28152 tablet_bootstrap.cc:492] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7: No bootstrap required, opened a new log
I20260929 19:14:02.223016 28152 ts_tablet_manager.cc:1397] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:14:02.224550 28149 raft_consensus.cc:348] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "827d104c02dc4b4ea9e3e619fbc20432" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46167 } } peers { permanent_uuid: "4a6da50914eb49338a1213cdf0388b3e" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35875 } } peers { permanent_uuid: "108b80712447401e8c55bfe2ca9069b7" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36733 } }
I20260929 19:14:02.224611 28150 raft_consensus.cc:348] T 89ef358e7b714a43898a8fb44ff3dca5 P 4a6da50914eb49338a1213cdf0388b3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "827d104c02dc4b4ea9e3e619fbc20432" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46167 } } peers { permanent_uuid: "4a6da50914eb49338a1213cdf0388b3e" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35875 } } peers { permanent_uuid: "108b80712447401e8c55bfe2ca9069b7" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36733 } }
I20260929 19:14:02.224725 28149 raft_consensus.cc:374] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:02.224764 28149 raft_consensus.cc:729] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 827d104c02dc4b4ea9e3e619fbc20432, State: Initialized, Role: FOLLOWER
I20260929 19:14:02.224766 28150 raft_consensus.cc:374] T 89ef358e7b714a43898a8fb44ff3dca5 P 4a6da50914eb49338a1213cdf0388b3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:02.224807 28150 raft_consensus.cc:729] T 89ef358e7b714a43898a8fb44ff3dca5 P 4a6da50914eb49338a1213cdf0388b3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a6da50914eb49338a1213cdf0388b3e, State: Initialized, Role: FOLLOWER
I20260929 19:14:02.224882 28149 consensus_queue.cc:260] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "827d104c02dc4b4ea9e3e619fbc20432" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46167 } } peers { permanent_uuid: "4a6da50914eb49338a1213cdf0388b3e" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35875 } } peers { permanent_uuid: "108b80712447401e8c55bfe2ca9069b7" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36733 } }
I20260929 19:14:02.224949 28150 consensus_queue.cc:260] T 89ef358e7b714a43898a8fb44ff3dca5 P 4a6da50914eb49338a1213cdf0388b3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "827d104c02dc4b4ea9e3e619fbc20432" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46167 } } peers { permanent_uuid: "4a6da50914eb49338a1213cdf0388b3e" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35875 } } peers { permanent_uuid: "108b80712447401e8c55bfe2ca9069b7" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36733 } }
I20260929 19:14:02.225157 28149 ts_tablet_manager.cc:1428] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:14:02.225248 28150 ts_tablet_manager.cc:1428] T 89ef358e7b714a43898a8fb44ff3dca5 P 4a6da50914eb49338a1213cdf0388b3e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:14:02.225289 27873 heartbeater.cc:503] Master 127.9.236.62:45503 was elected leader, sending a full tablet report...
I20260929 19:14:02.225421 28135 heartbeater.cc:503] Master 127.9.236.62:45503 was elected leader, sending a full tablet report...
I20260929 19:14:02.225893 28152 raft_consensus.cc:348] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "827d104c02dc4b4ea9e3e619fbc20432" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46167 } } peers { permanent_uuid: "4a6da50914eb49338a1213cdf0388b3e" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35875 } } peers { permanent_uuid: "108b80712447401e8c55bfe2ca9069b7" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36733 } }
I20260929 19:14:02.226018 28152 raft_consensus.cc:374] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:02.226038 28152 raft_consensus.cc:729] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 108b80712447401e8c55bfe2ca9069b7, State: Initialized, Role: FOLLOWER
I20260929 19:14:02.226143 28152 consensus_queue.cc:260] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "827d104c02dc4b4ea9e3e619fbc20432" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46167 } } peers { permanent_uuid: "4a6da50914eb49338a1213cdf0388b3e" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35875 } } peers { permanent_uuid: "108b80712447401e8c55bfe2ca9069b7" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36733 } }
I20260929 19:14:02.226398 28152 ts_tablet_manager.cc:1428] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:14:02.226503 28004 heartbeater.cc:503] Master 127.9.236.62:45503 was elected leader, sending a full tablet report...
I20260929 19:14:02.290143 28155 raft_consensus.cc:482] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:14:02.290263 28155 raft_consensus.cc:504] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "827d104c02dc4b4ea9e3e619fbc20432" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46167 } } peers { permanent_uuid: "4a6da50914eb49338a1213cdf0388b3e" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35875 } } peers { permanent_uuid: "108b80712447401e8c55bfe2ca9069b7" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36733 } }
I20260929 19:14:02.290586 28155 leader_election.cc:288] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a6da50914eb49338a1213cdf0388b3e (127.9.236.3:35875), 108b80712447401e8c55bfe2ca9069b7 (127.9.236.2:36733)
I20260929 19:14:02.294669 28148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:02.290895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:44899 (local address 127.9.236.2:36733)
0929 19:14:02.290954 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:02.290956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:02.291309 (+ 353us) server_negotiation.cc:407] Connection header received
0929 19:14:02.291380 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:02.291381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:02.291391 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:02.291452 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:02.291584 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.292270 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.293734 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.294313 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.294381 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:02.294392 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:02.294554 (+ 162us) server_negotiation.cc:299] Negotiation successful
0929 19:14:02.294592 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 19:14:02.294669 28147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:02.290916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:34257 (local address 127.9.236.3:35875)
0929 19:14:02.290981 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:02.290983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:02.290994 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:14:02.291080 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:02.291082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:02.291093 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:02.291159 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:02.291376 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.292137 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.293738 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.294300 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.294381 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:02.294392 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:02.294553 (+ 161us) server_negotiation.cc:299] Negotiation successful
0929 19:14:02.294592 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 19:14:02.294705 27875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:02.290822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:35875 (local address 127.9.236.1:34257)
0929 19:14:02.290920 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:02.290933 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:02.290983 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:02.291181 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:02.291182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:02.291184 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:02.291286 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:02.291289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.292239 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.292240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:02.293640 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:02.293644 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.294399 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.294400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:02.294476 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:02.294481 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:14:02.294539 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 19:14:02.294597 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":12}
I20260929 19:14:02.294710 28158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:02.290822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:36733 (local address 127.9.236.1:44899)
0929 19:14:02.291226 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:02.291237 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:02.291303 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:02.291464 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:02.291464 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:02.291466 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:02.291522 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:02.291523 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.292354 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.292355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:02.293644 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:02.293645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.294391 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.294392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:02.294476 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:02.294481 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:14:02.294539 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 19:14:02.294593 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":214,"thread_start_us":130,"threads_started":1}
I20260929 19:14:02.295207 27958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89ef358e7b714a43898a8fb44ff3dca5" candidate_uuid: "827d104c02dc4b4ea9e3e619fbc20432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "108b80712447401e8c55bfe2ca9069b7" is_pre_election: true
I20260929 19:14:02.295183 28089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89ef358e7b714a43898a8fb44ff3dca5" candidate_uuid: "827d104c02dc4b4ea9e3e619fbc20432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a6da50914eb49338a1213cdf0388b3e" is_pre_election: true
I20260929 19:14:02.295384 27958 raft_consensus.cc:2454] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 827d104c02dc4b4ea9e3e619fbc20432 in term 0.
I20260929 19:14:02.295385 28089 raft_consensus.cc:2454] T 89ef358e7b714a43898a8fb44ff3dca5 P 4a6da50914eb49338a1213cdf0388b3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 827d104c02dc4b4ea9e3e619fbc20432 in term 0.
I20260929 19:14:02.295652 27761 leader_election.cc:302] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a6da50914eb49338a1213cdf0388b3e, 827d104c02dc4b4ea9e3e619fbc20432; no voters:
I20260929 19:14:02.295819 28155 raft_consensus.cc:2781] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:14:02.295907 28155 raft_consensus.cc:482] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:14:02.295923 28155 raft_consensus.cc:3037] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:02.296679 28155 raft_consensus.cc:504] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "827d104c02dc4b4ea9e3e619fbc20432" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46167 } } peers { permanent_uuid: "4a6da50914eb49338a1213cdf0388b3e" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35875 } } peers { permanent_uuid: "108b80712447401e8c55bfe2ca9069b7" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36733 } }
I20260929 19:14:02.296821 28155 leader_election.cc:288] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432 [CANDIDATE]: Term 1 election: Requested vote from peers 4a6da50914eb49338a1213cdf0388b3e (127.9.236.3:35875), 108b80712447401e8c55bfe2ca9069b7 (127.9.236.2:36733)
I20260929 19:14:02.297000 27958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89ef358e7b714a43898a8fb44ff3dca5" candidate_uuid: "827d104c02dc4b4ea9e3e619fbc20432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "108b80712447401e8c55bfe2ca9069b7"
I20260929 19:14:02.297029 28089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89ef358e7b714a43898a8fb44ff3dca5" candidate_uuid: "827d104c02dc4b4ea9e3e619fbc20432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a6da50914eb49338a1213cdf0388b3e"
I20260929 19:14:02.297138 28089 raft_consensus.cc:3037] T 89ef358e7b714a43898a8fb44ff3dca5 P 4a6da50914eb49338a1213cdf0388b3e [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:02.297127 27958 raft_consensus.cc:3037] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:02.298043 28089 raft_consensus.cc:2454] T 89ef358e7b714a43898a8fb44ff3dca5 P 4a6da50914eb49338a1213cdf0388b3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 827d104c02dc4b4ea9e3e619fbc20432 in term 1.
I20260929 19:14:02.298043 27958 raft_consensus.cc:2454] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 827d104c02dc4b4ea9e3e619fbc20432 in term 1.
I20260929 19:14:02.298229 27761 leader_election.cc:302] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a6da50914eb49338a1213cdf0388b3e, 827d104c02dc4b4ea9e3e619fbc20432; no voters:
I20260929 19:14:02.298374 28155 raft_consensus.cc:2781] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:02.298565 28155 raft_consensus.cc:686] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432 [term 1 LEADER]: Becoming Leader. State: Replica: 827d104c02dc4b4ea9e3e619fbc20432, State: Running, Role: LEADER
I20260929 19:14:02.298703 28155 consensus_queue.cc:237] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "827d104c02dc4b4ea9e3e619fbc20432" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46167 } } peers { permanent_uuid: "4a6da50914eb49338a1213cdf0388b3e" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35875 } } peers { permanent_uuid: "108b80712447401e8c55bfe2ca9069b7" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36733 } }
I20260929 19:14:02.299715 27684 catalog_manager.cc:5495] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 827d104c02dc4b4ea9e3e619fbc20432 (127.9.236.1). New cstate: current_term: 1 leader_uuid: "827d104c02dc4b4ea9e3e619fbc20432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "827d104c02dc4b4ea9e3e619fbc20432" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 46167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a6da50914eb49338a1213cdf0388b3e" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "108b80712447401e8c55bfe2ca9069b7" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36733 } health_report { overall_health: UNKNOWN } } }
I20260929 19:14:02.306931 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:02.304567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37768 (local address 127.9.236.1:46167)
0929 19:14:02.304621 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:02.304623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:02.304628 (+ 5us) server_negotiation.cc:407] Connection header received
0929 19:14:02.304660 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:02.304661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:02.304677 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:02.304741 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:02.304870 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.305407 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.306197 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:02.306393 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:02.306450 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:02.306594 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:02.306710 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:02.306788 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:02.306805 (+ 17us) server_negotiation.cc:299] Negotiation successful
0929 19:14:02.306826 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 19:14:02.309037 28089 raft_consensus.cc:1261] T 89ef358e7b714a43898a8fb44ff3dca5 P 4a6da50914eb49338a1213cdf0388b3e [term 1 FOLLOWER]: Refusing update from remote peer 827d104c02dc4b4ea9e3e619fbc20432: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260929 19:14:02.309063 28005 tablet.cc:2369] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:14:02.309149 27958 raft_consensus.cc:1261] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7 [term 1 FOLLOWER]: Refusing update from remote peer 827d104c02dc4b4ea9e3e619fbc20432: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:14:02.309464 28159 consensus_queue.cc:1035] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a6da50914eb49338a1213cdf0388b3e" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 35875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:14:02.309608 28155 consensus_queue.cc:1035] T 89ef358e7b714a43898a8fb44ff3dca5 P 827d104c02dc4b4ea9e3e619fbc20432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "108b80712447401e8c55bfe2ca9069b7" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 36733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:14:02.312093 28162 mvcc.cc:205] Tried to move back new op lower bound from 7334745056494936064 to 7334745056456339456. Current Snapshot: MvccSnapshot[applied={T|T < 7334745056494936064}]
I20260929 19:14:02.312285 28164 mvcc.cc:205] Tried to move back new op lower bound from 7334745056494936064 to 7334745056456339456. Current Snapshot: MvccSnapshot[applied={T|T < 7334745056494936064}]
I20260929 19:14:02.312726 28163 mvcc.cc:205] Tried to move back new op lower bound from 7334745056494936064 to 7334745056456339456. Current Snapshot: MvccSnapshot[applied={T|T < 7334745056494936064}]
I20260929 19:14:02.314697 27684 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:14:02.314810 27684 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:14:02.315636 27684 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42cdb2aa57444a4585a56703ba10cbc3: Sending DeleteTablet for 3 replicas of tablet 89ef358e7b714a43898a8fb44ff3dca5
I20260929 19:14:02.315989 28069 tablet_service.cc:1514] Processing DeleteTablet for tablet 89ef358e7b714a43898a8fb44ff3dca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:14:02 UTC) from {username='slave'} at 127.0.0.1:41380
I20260929 19:14:02.316079 27938 tablet_service.cc:1514] Processing DeleteTablet for tablet 89ef358e7b714a43898a8fb44ff3dca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:14:02 UTC) from {username='slave'} at 127.0.0.1:34672
I20260929 19:14:02.316251 27807 tablet_service.cc:1514] Processing DeleteTablet for tablet 89ef358e7b714a43898a8fb44ff3dca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:14:02 UTC) from {username='slave'} at 127.0.0.1:37754
I20260929 19:14:02.316309 28172 tablet_replica.cc:330] T 89ef358e7b714a43898a8fb44ff3dca5 P 4a6da50914eb49338a1213cdf0388b3e: stopping tablet replica
I20260929 19:14:02.316396 28173 tablet_replica.cc:330] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7: stopping tablet replica
I20260929 19:14:02.316430 28172 raft_consensus.cc:2229] T 89ef358e7b714a43898a8fb44ff3dca5 P 4a6da50914eb49338a1213cdf0388b3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:14:02.316570 28172 raft_consensus.cc:2258] T 89ef358e7b714a43898a8fb44ff3dca5 P 4a6da50914eb49338a1213cdf0388b3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:14:02.316570 28173 raft_consensus.cc:2229] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:14:02.316720 28173 raft_consensus.cc:2258] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:14:02.316720 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 27745
I20260929 19:14:02.318114 28172 ts_tablet_manager.cc:1905] T 89ef358e7b714a43898a8fb44ff3dca5 P 4a6da50914eb49338a1213cdf0388b3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:14:02.318114 28173 ts_tablet_manager.cc:1905] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:14:02.320168 28172 ts_tablet_manager.cc:1918] T 89ef358e7b714a43898a8fb44ff3dca5 P 4a6da50914eb49338a1213cdf0388b3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:14:02.320168 28173 ts_tablet_manager.cc:1918] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:14:02.320250 28172 log.cc:1196] T 89ef358e7b714a43898a8fb44ff3dca5 P 4a6da50914eb49338a1213cdf0388b3e: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/89ef358e7b714a43898a8fb44ff3dca5
I20260929 19:14:02.320366 28173 log.cc:1196] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/89ef358e7b714a43898a8fb44ff3dca5
I20260929 19:14:02.320528 28172 ts_tablet_manager.cc:1939] T 89ef358e7b714a43898a8fb44ff3dca5 P 4a6da50914eb49338a1213cdf0388b3e: Deleting consensus metadata
I20260929 19:14:02.320727 28173 ts_tablet_manager.cc:1939] T 89ef358e7b714a43898a8fb44ff3dca5 P 108b80712447401e8c55bfe2ca9069b7: Deleting consensus metadata
I20260929 19:14:02.321049 27670 catalog_manager.cc:4841] TS 4a6da50914eb49338a1213cdf0388b3e (127.9.236.3:35875): tablet 89ef358e7b714a43898a8fb44ff3dca5 (table test-table [id=4bef8efe1b6942d0b55c7ec2f7b17db9]) successfully deleted
I20260929 19:14:02.321218 27669 catalog_manager.cc:4841] TS 108b80712447401e8c55bfe2ca9069b7 (127.9.236.2:36733): tablet 89ef358e7b714a43898a8fb44ff3dca5 (table test-table [id=4bef8efe1b6942d0b55c7ec2f7b17db9]) successfully deleted
W20260929 19:14:02.322073 27669 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.236.1:46167 (error 108)
I20260929 19:14:02.322325 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 27876
I20260929 19:14:02.322976 28144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:02.322479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:46167 (local address 127.0.0.1:37770)
0929 19:14:02.322888 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:02.322940 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.1:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385}
W20260929 19:14:02.323225 27669 catalog_manager.cc:4584] TS 827d104c02dc4b4ea9e3e619fbc20432 (127.9.236.1:46167): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 89ef358e7b714a43898a8fb44ff3dca5: Network error: Client connection negotiation failed: client connection to 127.9.236.1:46167: connect: Connection refused (error 111)
I20260929 19:14:02.326839 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 28007
I20260929 19:14:02.331619 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 27654
2026-09-29T19:14:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (728 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:02 dist-test-slave-wftq krb5kdc[28178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:02 dist-test-slave-wftq krb5kdc[28178](info): set up 2 sockets
Sep 29 19:14:02 dist-test-slave-wftq krb5kdc[28178](info): commencing operation
krb5kdc: starting...
W20260929 19:14:03.284668 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:03 dist-test-slave-wftq krb5kdc[28178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:14:03.405295 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:40819
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:41709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:40819
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:03.491995 28191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:03.492216 28191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:03.492236 28191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:03.493459 28191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:03.493503 28191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:03.493517 28191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:03.493528 28191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:03.493539 28191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:03.495042 28191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:41709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:40819
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:40819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:03.495272 28191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:03.495502 28191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:03.500479 28197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:03.500497 28196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:03.500479 28199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:03.500722 28191 server_base.cc:1034] running on GCE node
I20260929 19:14:03.501017 28191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:03.501340 28191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:03.502483 28191 hybrid_clock.cc:648] HybridClock initialized: now 1790709243502476 us; error 30 us; skew 500 ppm
Sep 29 19:14:03 dist-test-slave-wftq krb5kdc[28178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709243, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:03.505075 28191 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:14:03.505584 28191 webserver.cc:460] Webserver started at http://127.9.236.62:42679/ using document root <none> and password file <none>
I20260929 19:14:03.505822 28191 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:03.505877 28191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:03.506021 28191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:03.507184 28191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "6a2317f9ac1e4a3cb0a3b86679a58592"
format_stamp: "Formatted at 2026-09-29 19:14:03 on dist-test-slave-wftq"
server_key: "a8b2dc37d7201ed585717eaf42bf99a5"
server_key_iv: "eeacbb5fb7552e334e27a73e0cdcab98"
server_key_version: "encryptionkey@0"
I20260929 19:14:03.507539 28191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "6a2317f9ac1e4a3cb0a3b86679a58592"
format_stamp: "Formatted at 2026-09-29 19:14:03 on dist-test-slave-wftq"
server_key: "a8b2dc37d7201ed585717eaf42bf99a5"
server_key_iv: "eeacbb5fb7552e334e27a73e0cdcab98"
server_key_version: "encryptionkey@0"
I20260929 19:14:03.508971 28191 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260929 19:14:03.509828 28206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:03.510051 28191 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:03.510144 28191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "6a2317f9ac1e4a3cb0a3b86679a58592"
format_stamp: "Formatted at 2026-09-29 19:14:03 on dist-test-slave-wftq"
server_key: "a8b2dc37d7201ed585717eaf42bf99a5"
server_key_iv: "eeacbb5fb7552e334e27a73e0cdcab98"
server_key_version: "encryptionkey@0"
I20260929 19:14:03.510217 28191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:03.531140 28191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:03.533336 28191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:03.533476 28191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:03.538010 28191 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:40819
I20260929 19:14:03.538069 28258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:40819 every 8 connection(s)
I20260929 19:14:03.538419 28191 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:03.539347 28259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:03.540443 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 28191
I20260929 19:14:03.540549 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:03.540735 10160 external_mini_cluster.cc:1428] Setting key 8298f61dfd0a34ffaf5b54856895b38f
I20260929 19:14:03.542025 28259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592: Bootstrap starting.
I20260929 19:14:03.542604 28259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:03.542867 28259 log.cc:824] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:03.543476 28259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592: No bootstrap required, opened a new log
I20260929 19:14:03.545639 28259 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a2317f9ac1e4a3cb0a3b86679a58592" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40819 } }
I20260929 19:14:03.545825 28259 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:03.545861 28259 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a2317f9ac1e4a3cb0a3b86679a58592, State: Initialized, Role: FOLLOWER
I20260929 19:14:03.546001 28259 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a2317f9ac1e4a3cb0a3b86679a58592" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40819 } }
I20260929 19:14:03.546082 28259 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:03.546103 28259 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:03.546137 28259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:03.546720 28259 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a2317f9ac1e4a3cb0a3b86679a58592" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40819 } }
I20260929 19:14:03.546804 28259 leader_election.cc:302] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a2317f9ac1e4a3cb0a3b86679a58592; no voters:
I20260929 19:14:03.546959 28259 leader_election.cc:288] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:03.547001 28264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:03.547120 28264 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592 [term 1 LEADER]: Becoming Leader. State: Replica: 6a2317f9ac1e4a3cb0a3b86679a58592, State: Running, Role: LEADER
I20260929 19:14:03.547214 28264 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a2317f9ac1e4a3cb0a3b86679a58592" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40819 } }
I20260929 19:14:03.547264 28259 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592 [sys.catalog]: configured and running, proceeding with master startup.
Sep 29 19:14:03 dist-test-slave-wftq krb5kdc[28178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:14:03.547647 28264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a2317f9ac1e4a3cb0a3b86679a58592. Latest consensus state: current_term: 1 leader_uuid: "6a2317f9ac1e4a3cb0a3b86679a58592" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a2317f9ac1e4a3cb0a3b86679a58592" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40819 } } }
I20260929 19:14:03.547629 28266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a2317f9ac1e4a3cb0a3b86679a58592" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a2317f9ac1e4a3cb0a3b86679a58592" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 40819 } } }
I20260929 19:14:03.547725 28264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:03.547740 28266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:03.548112 28271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:03.548659 28271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:03.550621 28271 catalog_manager.cc:1347] Generated new cluster ID: 4c8e18c5109a4192875c6428094dae09
I20260929 19:14:03.550710 28271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:03.552508 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:03.541317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51184 (local address 127.9.236.62:40819)
0929 19:14:03.541530 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:03.541539 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:03.541571 (+ 32us) server_negotiation.cc:407] Connection header received
0929 19:14:03.542134 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:03.542143 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:03.542212 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:03.542359 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:03.543082 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:03.544137 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:03.544880 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:03.545369 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:03.545482 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:03.548185 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:03.548191 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:03.548191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:03.548206 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:03.550339 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:03.550763 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:03.550766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:03.550767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:03.550889 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:03.551081 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:03.551082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:03.551083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:03.551393 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:03.551558 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:03.552112 (+ 554us) server_negotiation.cc:299] Negotiation successful
0929 19:14:03.552268 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":102,"threads_started":1}
I20260929 19:14:03.561558 28271 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:03.562108 28271 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:03.566977 28271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592: Generated new TSK 0
I20260929 19:14:03.567140 28271 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.236.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
I20260929 19:14:03.597971 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:40819
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:41709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:03.680537 28285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:03.680765 28285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:03.680789 28285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:03.681965 28285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:03.682011 28285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:03.682039 28285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:14:03.683439 28285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:41709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:40819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:03.683652 28285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:03.683841 28285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:03.688746 28290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:03.688874 28293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:03.688927 28291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:03.689077 28285 server_base.cc:1034] running on GCE node
I20260929 19:14:03.689319 28285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:03.689553 28285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:03.690696 28285 hybrid_clock.cc:648] HybridClock initialized: now 1790709243690679 us; error 35 us; skew 500 ppm
Sep 29 19:14:03 dist-test-slave-wftq krb5kdc[28178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709243, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:03.693238 28285 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:14:03.693660 28285 webserver.cc:460] Webserver started at http://127.9.236.1:40033/ using document root <none> and password file <none>
I20260929 19:14:03.693863 28285 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:03.693912 28285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:03.694032 28285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:03.695061 28285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "f6cf7d4003f44ed79106c606f32d44c6"
format_stamp: "Formatted at 2026-09-29 19:14:03 on dist-test-slave-wftq"
server_key: "f4d273d34f0e14ab41dfd1d583b53ff4"
server_key_iv: "96b94f6c7430ddc97036424d67e288d7"
server_key_version: "encryptionkey@0"
I20260929 19:14:03.695452 28285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "f6cf7d4003f44ed79106c606f32d44c6"
format_stamp: "Formatted at 2026-09-29 19:14:03 on dist-test-slave-wftq"
server_key: "f4d273d34f0e14ab41dfd1d583b53ff4"
server_key_iv: "96b94f6c7430ddc97036424d67e288d7"
server_key_version: "encryptionkey@0"
I20260929 19:14:03.696774 28285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:14:03.697572 28300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:03.697765 28285 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 19:14:03.697860 28285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "f6cf7d4003f44ed79106c606f32d44c6"
format_stamp: "Formatted at 2026-09-29 19:14:03 on dist-test-slave-wftq"
server_key: "f4d273d34f0e14ab41dfd1d583b53ff4"
server_key_iv: "96b94f6c7430ddc97036424d67e288d7"
server_key_version: "encryptionkey@0"
I20260929 19:14:03.697937 28285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:03.710549 28285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:03.712735 28285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:03.712857 28285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:03.713138 28285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:14:03.713475 28285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:14:03.713516 28285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:03.713549 28285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:14:03.713567 28285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:03.720309 28285 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:44225
I20260929 19:14:03.720397 28413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:44225 every 8 connection(s)
I20260929 19:14:03.720700 28285 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:14:03.722411 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 28285
I20260929 19:14:03.722505 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:14:03.722666 10160 external_mini_cluster.cc:1428] Setting key def859f965243e816bf5fbffa99f15de
Sep 29 19:14:03 dist-test-slave-wftq krb5kdc[28178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709243, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.236.2@KRBTEST.COM; defaulting to no policy
I20260929 19:14:03.730225 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:03.721496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:39661 (local address 127.9.236.62:40819)
0929 19:14:03.721586 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:03.721589 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:03.721781 (+ 192us) server_negotiation.cc:407] Connection header received
0929 19:14:03.722366 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:03.722368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:03.722382 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:03.722442 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:03.723481 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:03.724032 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:03.725043 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:03.725320 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:03.725378 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:03.727537 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:03.727539 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:03.727539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:03.727547 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:03.728844 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:03.729266 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:03.729267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:03.729267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:03.729321 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:03.729537 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:03.729538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:03.729539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:03.729661 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:03.729724 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:03.730021 (+ 297us) server_negotiation.cc:299] Negotiation successful
0929 19:14:03.730125 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 19:14:03.730387 28416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:03.721448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:40819 (local address 127.9.236.1:39661)
0929 19:14:03.721625 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:03.721666 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:03.722251 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:03.722521 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:03.722526 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:03.723163 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:03.723381 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:03.723386 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:03.724115 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:03.724117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:03.724960 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:03.724963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:03.725392 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:03.725393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:03.725527 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:03.726022 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:03.726031 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:03.727452 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:03.728918 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:03.728925 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:03.728936 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:03.729194 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:03.729372 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:03.729373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:03.729373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:03.729481 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:03.729713 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:03.729720 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:03.729863 (+ 143us) client_negotiation.cc:769] Sending connection context
0929 19:14:03.730000 (+ 137us) client_negotiation.cc:240] Negotiation successful
0929 19:14:03.730164 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":97,"threads_started":1}
I20260929 19:14:03.730974 28414 heartbeater.cc:347] Connected to a master server at 127.9.236.62:40819
I20260929 19:14:03.731088 28414 heartbeater.cc:464] Registering TS with master...
I20260929 19:14:03.731287 28414 heartbeater.cc:511] Master 127.9.236.62:40819 requested a full tablet report, sending...
I20260929 19:14:03.731837 28223 ts_manager.cc:194] Registered new tserver with Master: f6cf7d4003f44ed79106c606f32d44c6 (127.9.236.1:44225)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.2@KRBTEST.COM" created.
I20260929 19:14:03.732544 28223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:39661
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
I20260929 19:14:03.747660 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:40819
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:41709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:03.825814 28419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:03.826020 28419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:03.826048 28419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:03.827169 28419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:03.827212 28419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:03.827245 28419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:14:03.828672 28419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:41709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:40819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:03.828907 28419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:03.829128 28419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:03.834081 28425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:03.834079 28427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:03.834084 28424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:03.834378 28419 server_base.cc:1034] running on GCE node
I20260929 19:14:03.834590 28419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:03.834787 28419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:03.835911 28419 hybrid_clock.cc:648] HybridClock initialized: now 1790709243835894 us; error 35 us; skew 500 ppm
Sep 29 19:14:03 dist-test-slave-wftq krb5kdc[28178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709243, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:03.838347 28419 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:14:03.838809 28419 webserver.cc:460] Webserver started at http://127.9.236.2:37213/ using document root <none> and password file <none>
I20260929 19:14:03.838996 28419 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:03.839042 28419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:03.839156 28419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:03.840137 28419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "f33fb5f0f8394f23948b2127da5a2777"
format_stamp: "Formatted at 2026-09-29 19:14:03 on dist-test-slave-wftq"
server_key: "fe1ba70956dcaad894769a49c62df7e7"
server_key_iv: "15668ac232e34e21a60eb67a64d6633c"
server_key_version: "encryptionkey@0"
I20260929 19:14:03.840442 28419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "f33fb5f0f8394f23948b2127da5a2777"
format_stamp: "Formatted at 2026-09-29 19:14:03 on dist-test-slave-wftq"
server_key: "fe1ba70956dcaad894769a49c62df7e7"
server_key_iv: "15668ac232e34e21a60eb67a64d6633c"
server_key_version: "encryptionkey@0"
I20260929 19:14:03.841688 28419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:14:03.842446 28434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:03.842639 28419 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:03.842749 28419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "f33fb5f0f8394f23948b2127da5a2777"
format_stamp: "Formatted at 2026-09-29 19:14:03 on dist-test-slave-wftq"
server_key: "fe1ba70956dcaad894769a49c62df7e7"
server_key_iv: "15668ac232e34e21a60eb67a64d6633c"
server_key_version: "encryptionkey@0"
I20260929 19:14:03.842809 28419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:03.860492 28419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:03.862468 28419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:03.862603 28419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:03.862844 28419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:14:03.863149 28419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:14:03.863183 28419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:03.863215 28419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:14:03.863232 28419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:03.869514 28419 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:33723
I20260929 19:14:03.869581 28547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:33723 every 8 connection(s)
I20260929 19:14:03.869866 28419 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:14:03.872988 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 28419
I20260929 19:14:03.873092 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:14:03.873310 10160 external_mini_cluster.cc:1428] Setting key d4318d237cf680f2be5cb063ec07ddcd
Sep 29 19:14:03 dist-test-slave-wftq krb5kdc[28178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709243, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:14:03.880789 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:03.870642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:57171 (local address 127.9.236.62:40819)
0929 19:14:03.870704 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:03.870707 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:03.870911 (+ 204us) server_negotiation.cc:407] Connection header received
0929 19:14:03.871548 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:03.871549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:03.871563 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:03.871625 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:03.872703 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:03.873484 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:03.874973 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:03.875177 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:03.875233 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:03.877777 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:03.877779 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:03.877779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:03.877788 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:03.879387 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:03.879803 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:03.879804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:03.879805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:03.879874 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:03.880112 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:03.880113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:03.880113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:03.880259 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:03.880334 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:03.880588 (+ 254us) server_negotiation.cc:299] Negotiation successful
0929 19:14:03.880678 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 19:14:03.880829 28550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:03.870564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:40819 (local address 127.9.236.2:57171)
0929 19:14:03.870762 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:03.870809 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:03.871438 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:03.871705 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:03.871711 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:03.872360 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:03.872601 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:03.872606 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:03.873634 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:03.873636 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:03.874859 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:03.874864 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:03.875260 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:03.875261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:03.875400 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:03.876057 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:03.876065 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:03.877678 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:03.879486 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:03.879491 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:03.879493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:03.879724 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:03.879950 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:03.879951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:03.879951 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:03.880045 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:03.880332 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:03.880337 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:03.880454 (+ 117us) client_negotiation.cc:769] Sending connection context
0929 19:14:03.880539 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 19:14:03.880666 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":98,"threads_started":1}
I20260929 19:14:03.881636 28548 heartbeater.cc:347] Connected to a master server at 127.9.236.62:40819
I20260929 19:14:03.881770 28548 heartbeater.cc:464] Registering TS with master...
I20260929 19:14:03.882025 28548 heartbeater.cc:511] Master 127.9.236.62:40819 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.236.3@KRBTEST.COM; defaulting to no policy
I20260929 19:14:03.882521 28223 ts_manager.cc:194] Registered new tserver with Master: f33fb5f0f8394f23948b2127da5a2777 (127.9.236.2:33723)
I20260929 19:14:03.882926 28223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:57171
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
I20260929 19:14:03.899535 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:40819
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:41709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:03.986244 28553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:03.986495 28553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:03.986529 28553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:03.987767 28553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:03.987810 28553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:03.987847 28553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:14:03.989415 28553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:41709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:40819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:03.989687 28553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:03.989934 28553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:03.995134 28559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:03.995157 28561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:03.995134 28558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:03.995406 28553 server_base.cc:1034] running on GCE node
I20260929 19:14:03.995627 28553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:03.995872 28553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:03.997025 28553 hybrid_clock.cc:648] HybridClock initialized: now 1790709243997005 us; error 42 us; skew 500 ppm
Sep 29 19:14:03 dist-test-slave-wftq krb5kdc[28178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709243, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:03.999698 28553 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:14:04.000227 28553 webserver.cc:460] Webserver started at http://127.9.236.3:44689/ using document root <none> and password file <none>
I20260929 19:14:04.000451 28553 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:04.000510 28553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:04.000639 28553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:04.001859 28553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "328f0a45507b4cd1a079c4a4e928e71f"
format_stamp: "Formatted at 2026-09-29 19:14:03 on dist-test-slave-wftq"
server_key: "ebb3c1285febdc0c92ed24a5ac365a24"
server_key_iv: "52c040b17d5f046e46f69d0be39141b6"
server_key_version: "encryptionkey@0"
I20260929 19:14:04.002206 28553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "328f0a45507b4cd1a079c4a4e928e71f"
format_stamp: "Formatted at 2026-09-29 19:14:03 on dist-test-slave-wftq"
server_key: "ebb3c1285febdc0c92ed24a5ac365a24"
server_key_iv: "52c040b17d5f046e46f69d0be39141b6"
server_key_version: "encryptionkey@0"
I20260929 19:14:04.003575 28553 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:14:04.004446 28568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:04.004647 28553 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:04.004733 28553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "328f0a45507b4cd1a079c4a4e928e71f"
format_stamp: "Formatted at 2026-09-29 19:14:03 on dist-test-slave-wftq"
server_key: "ebb3c1285febdc0c92ed24a5ac365a24"
server_key_iv: "52c040b17d5f046e46f69d0be39141b6"
server_key_version: "encryptionkey@0"
I20260929 19:14:04.004794 28553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:04.027940 28553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:04.030244 28553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:04.030388 28553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:04.030654 28553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:14:04.031008 28553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:14:04.031042 28553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:04.031073 28553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:14:04.031096 28553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:04.038200 28553 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:37265
I20260929 19:14:04.038257 28681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:37265 every 8 connection(s)
I20260929 19:14:04.038601 28553 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:14:04.044886 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 28553
I20260929 19:14:04.044996 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
Sep 29 19:14:04 dist-test-slave-wftq krb5kdc[28178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709243, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:14:04.045207 10160 external_mini_cluster.cc:1428] Setting key c199eb0275c1f626b8c70e8f861c700e
I20260929 19:14:04.048614 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.039465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:58533 (local address 127.9.236.62:40819)
0929 19:14:04.039536 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:04.039539 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:04.039634 (+ 95us) server_negotiation.cc:407] Connection header received
0929 19:14:04.040211 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:04.040212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:04.040225 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:04.040300 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:04.041341 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.041932 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.042954 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.043157 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.043225 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.045674 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:04.045676 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:04.045677 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:04.045687 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:04.047198 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:04.047598 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:04.047599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:04.047599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:04.047650 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:04.047876 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:04.047877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:04.047877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:04.048046 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:04.048104 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:04.048399 (+ 295us) server_negotiation.cc:299] Negotiation successful
0929 19:14:04.048492 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 19:14:04.048714 28684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.039323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:40819 (local address 127.9.236.3:58533)
0929 19:14:04.039515 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:04.039559 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:04.040087 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:04.040369 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:04.040372 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:04.040987 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:04.041242 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.041247 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.042021 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.042022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.042850 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.042852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.043242 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.043242 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.043368 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.043997 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:04.044006 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:04.045571 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:04.047265 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:04.047272 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:04.047275 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:04.047528 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:04.047707 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:04.047708 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:04.047708 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:04.047812 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:04.048119 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:04.048125 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:04.048255 (+ 130us) client_negotiation.cc:769] Sending connection context
0929 19:14:04.048344 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 19:14:04.048497 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":95,"threads_started":1}
I20260929 19:14:04.049237 28682 heartbeater.cc:347] Connected to a master server at 127.9.236.62:40819
I20260929 19:14:04.049400 28682 heartbeater.cc:464] Registering TS with master...
I20260929 19:14:04.049664 28682 heartbeater.cc:511] Master 127.9.236.62:40819 requested a full tablet report, sending...
I20260929 19:14:04.050088 28223 ts_manager.cc:194] Registered new tserver with Master: 328f0a45507b4cd1a079c4a4e928e71f (127.9.236.3:37265)
I20260929 19:14:04.050496 28223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:58533
I20260929 19:14:04.057138 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:14:04.064682 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.058657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51198 (local address 127.9.236.62:40819)
0929 19:14:04.058736 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:04.058738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:04.058798 (+ 60us) server_negotiation.cc:407] Connection header received
0929 19:14:04.058863 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:04.058864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:04.058872 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:04.058910 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:04.059478 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.059965 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.060726 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.060908 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.060960 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.062292 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:04.062293 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:04.062294 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:04.062300 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:04.063552 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:04.063941 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:04.063941 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:04.063942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:04.063991 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:04.064226 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:04.064227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:04.064227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:04.064343 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:04.064430 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:04.064558 (+ 128us) server_negotiation.cc:299] Negotiation successful
0929 19:14:04.064605 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 19:14:04.066526 28223 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51198:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:14:04.068361 28223 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:14:04.075558 28696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.071063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:37265 (local address 127.0.0.1:36104)
0929 19:14:04.071642 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:04.071652 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:04.071697 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:04.071909 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:04.071911 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:04.071913 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:04.072006 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.072008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.072977 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.072978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.074347 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.074350 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.075256 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.075257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.075331 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.075344 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 19:14:04.075419 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 19:14:04.075479 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"mutex_wait_us":3,"thread_start_us":52,"threads_started":1}
I20260929 19:14:04.075604 28698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.071468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36104 (local address 127.9.236.3:37265)
0929 19:14:04.071755 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:04.071759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:04.071769 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:14:04.071812 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:04.071814 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:04.071828 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:04.071911 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:04.072091 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.072894 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.074445 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.075164 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.075242 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.075308 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:04.075443 (+ 135us) server_negotiation.cc:299] Negotiation successful
0929 19:14:04.075496 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":149,"threads_started":1}
I20260929 19:14:04.075716 28693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.071297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:44225 (local address 127.0.0.1:60882)
0929 19:14:04.072004 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:04.072015 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:04.072071 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:04.072292 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:04.072293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:04.072296 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:04.072332 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.072334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.073612 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.073613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.074556 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.074559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.075423 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.075424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.075482 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.075485 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:14:04.075527 (+ 42us) client_negotiation.cc:240] Negotiation successful
0929 19:14:04.075569 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"mutex_wait_us":2,"thread_start_us":80,"threads_started":1}
I20260929 19:14:04.075757 28697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.071474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60882 (local address 127.9.236.1:44225)
0929 19:14:04.071623 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:04.071628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:04.072126 (+ 498us) server_negotiation.cc:407] Connection header received
0929 19:14:04.072177 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:04.072179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:04.072193 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:04.072274 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:04.072405 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.073497 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.074664 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.075253 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.075324 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.075386 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:04.075610 (+ 224us) server_negotiation.cc:299] Negotiation successful
0929 19:14:04.075658 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
I20260929 19:14:04.076684 28694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.071063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:33723 (local address 127.0.0.1:36762)
0929 19:14:04.071243 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:04.071286 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:04.071356 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:04.071887 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:04.071891 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:04.071895 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:04.071997 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.072000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.073703 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.073705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.074743 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.074746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.075619 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.076490 (+ 871us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.076553 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.076559 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 19:14:04.076588 (+ 29us) client_negotiation.cc:240] Negotiation successful
0929 19:14:04.076629 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
I20260929 19:14:04.076578 28348 tablet_service.cc:1467] Processing CreateTablet for tablet be797b99d7664887ac07a0d1267f975e (DEFAULT_TABLE table=test-table [id=71d817f684f04e8e8522a17232454816]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:14:04.076897 28616 tablet_service.cc:1467] Processing CreateTablet for tablet be797b99d7664887ac07a0d1267f975e (DEFAULT_TABLE table=test-table [id=71d817f684f04e8e8522a17232454816]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:14:04.077003 28348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be797b99d7664887ac07a0d1267f975e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:04.077083 28695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.071181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36762 (local address 127.9.236.2:33723)
0929 19:14:04.071509 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:04.071515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:04.071529 (+ 14us) server_negotiation.cc:407] Connection header received
0929 19:14:04.071727 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:04.071730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:04.071752 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:04.072094 (+ 342us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:04.072182 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.073463 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.074832 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.075504 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.076859 (+ 1355us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.076927 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:04.076952 (+ 25us) server_negotiation.cc:299] Negotiation successful
0929 19:14:04.077004 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":92,"threads_started":1}
I20260929 19:14:04.077265 28616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be797b99d7664887ac07a0d1267f975e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:04.077706 28482 tablet_service.cc:1467] Processing CreateTablet for tablet be797b99d7664887ac07a0d1267f975e (DEFAULT_TABLE table=test-table [id=71d817f684f04e8e8522a17232454816]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:14:04.078035 28482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be797b99d7664887ac07a0d1267f975e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:04.080662 28699 tablet_bootstrap.cc:492] T be797b99d7664887ac07a0d1267f975e P 328f0a45507b4cd1a079c4a4e928e71f: Bootstrap starting.
I20260929 19:14:04.081235 28700 tablet_bootstrap.cc:492] T be797b99d7664887ac07a0d1267f975e P f6cf7d4003f44ed79106c606f32d44c6: Bootstrap starting.
I20260929 19:14:04.081470 28701 tablet_bootstrap.cc:492] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777: Bootstrap starting.
I20260929 19:14:04.081555 28699 tablet_bootstrap.cc:654] T be797b99d7664887ac07a0d1267f975e P 328f0a45507b4cd1a079c4a4e928e71f: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:04.082072 28699 log.cc:824] T be797b99d7664887ac07a0d1267f975e P 328f0a45507b4cd1a079c4a4e928e71f: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:04.082080 28700 tablet_bootstrap.cc:654] T be797b99d7664887ac07a0d1267f975e P f6cf7d4003f44ed79106c606f32d44c6: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:04.082144 28701 tablet_bootstrap.cc:654] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:04.082670 28700 log.cc:824] T be797b99d7664887ac07a0d1267f975e P f6cf7d4003f44ed79106c606f32d44c6: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:04.082670 28701 log.cc:824] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:04.082885 28699 tablet_bootstrap.cc:492] T be797b99d7664887ac07a0d1267f975e P 328f0a45507b4cd1a079c4a4e928e71f: No bootstrap required, opened a new log
I20260929 19:14:04.082967 28699 ts_tablet_manager.cc:1397] T be797b99d7664887ac07a0d1267f975e P 328f0a45507b4cd1a079c4a4e928e71f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:14:04.083552 28701 tablet_bootstrap.cc:492] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777: No bootstrap required, opened a new log
I20260929 19:14:04.083647 28701 ts_tablet_manager.cc:1397] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:14:04.083776 28700 tablet_bootstrap.cc:492] T be797b99d7664887ac07a0d1267f975e P f6cf7d4003f44ed79106c606f32d44c6: No bootstrap required, opened a new log
I20260929 19:14:04.083841 28700 ts_tablet_manager.cc:1397] T be797b99d7664887ac07a0d1267f975e P f6cf7d4003f44ed79106c606f32d44c6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 19:14:04.084726 28699 raft_consensus.cc:348] T be797b99d7664887ac07a0d1267f975e P 328f0a45507b4cd1a079c4a4e928e71f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "328f0a45507b4cd1a079c4a4e928e71f" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 37265 } } peers { permanent_uuid: "f33fb5f0f8394f23948b2127da5a2777" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 33723 } } peers { permanent_uuid: "f6cf7d4003f44ed79106c606f32d44c6" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44225 } }
I20260929 19:14:04.084879 28699 raft_consensus.cc:374] T be797b99d7664887ac07a0d1267f975e P 328f0a45507b4cd1a079c4a4e928e71f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:04.084913 28699 raft_consensus.cc:729] T be797b99d7664887ac07a0d1267f975e P 328f0a45507b4cd1a079c4a4e928e71f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 328f0a45507b4cd1a079c4a4e928e71f, State: Initialized, Role: FOLLOWER
I20260929 19:14:04.085070 28699 consensus_queue.cc:260] T be797b99d7664887ac07a0d1267f975e P 328f0a45507b4cd1a079c4a4e928e71f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "328f0a45507b4cd1a079c4a4e928e71f" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 37265 } } peers { permanent_uuid: "f33fb5f0f8394f23948b2127da5a2777" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 33723 } } peers { permanent_uuid: "f6cf7d4003f44ed79106c606f32d44c6" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44225 } }
I20260929 19:14:04.085397 28682 heartbeater.cc:503] Master 127.9.236.62:40819 was elected leader, sending a full tablet report...
I20260929 19:14:04.085397 28699 ts_tablet_manager.cc:1428] T be797b99d7664887ac07a0d1267f975e P 328f0a45507b4cd1a079c4a4e928e71f: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260929 19:14:04.085711 28700 raft_consensus.cc:348] T be797b99d7664887ac07a0d1267f975e P f6cf7d4003f44ed79106c606f32d44c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "328f0a45507b4cd1a079c4a4e928e71f" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 37265 } } peers { permanent_uuid: "f33fb5f0f8394f23948b2127da5a2777" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 33723 } } peers { permanent_uuid: "f6cf7d4003f44ed79106c606f32d44c6" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44225 } }
I20260929 19:14:04.085896 28700 raft_consensus.cc:374] T be797b99d7664887ac07a0d1267f975e P f6cf7d4003f44ed79106c606f32d44c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:04.085942 28700 raft_consensus.cc:729] T be797b99d7664887ac07a0d1267f975e P f6cf7d4003f44ed79106c606f32d44c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6cf7d4003f44ed79106c606f32d44c6, State: Initialized, Role: FOLLOWER
I20260929 19:14:04.085884 28701 raft_consensus.cc:348] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "328f0a45507b4cd1a079c4a4e928e71f" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 37265 } } peers { permanent_uuid: "f33fb5f0f8394f23948b2127da5a2777" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 33723 } } peers { permanent_uuid: "f6cf7d4003f44ed79106c606f32d44c6" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44225 } }
I20260929 19:14:04.086047 28701 raft_consensus.cc:374] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:04.086076 28701 raft_consensus.cc:729] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f33fb5f0f8394f23948b2127da5a2777, State: Initialized, Role: FOLLOWER
I20260929 19:14:04.086072 28700 consensus_queue.cc:260] T be797b99d7664887ac07a0d1267f975e P f6cf7d4003f44ed79106c606f32d44c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "328f0a45507b4cd1a079c4a4e928e71f" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 37265 } } peers { permanent_uuid: "f33fb5f0f8394f23948b2127da5a2777" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 33723 } } peers { permanent_uuid: "f6cf7d4003f44ed79106c606f32d44c6" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44225 } }
I20260929 19:14:04.086207 28701 consensus_queue.cc:260] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "328f0a45507b4cd1a079c4a4e928e71f" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 37265 } } peers { permanent_uuid: "f33fb5f0f8394f23948b2127da5a2777" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 33723 } } peers { permanent_uuid: "f6cf7d4003f44ed79106c606f32d44c6" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44225 } }
I20260929 19:14:04.086371 28700 ts_tablet_manager.cc:1428] T be797b99d7664887ac07a0d1267f975e P f6cf7d4003f44ed79106c606f32d44c6: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260929 19:14:04.086464 28701 ts_tablet_manager.cc:1428] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:14:04.086501 28414 heartbeater.cc:503] Master 127.9.236.62:40819 was elected leader, sending a full tablet report...
I20260929 19:14:04.086620 28548 heartbeater.cc:503] Master 127.9.236.62:40819 was elected leader, sending a full tablet report...
I20260929 19:14:04.099469 28707 raft_consensus.cc:482] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:14:04.099591 28707 raft_consensus.cc:504] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "328f0a45507b4cd1a079c4a4e928e71f" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 37265 } } peers { permanent_uuid: "f33fb5f0f8394f23948b2127da5a2777" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 33723 } } peers { permanent_uuid: "f6cf7d4003f44ed79106c606f32d44c6" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44225 } }
I20260929 19:14:04.099973 28707 leader_election.cc:288] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 328f0a45507b4cd1a079c4a4e928e71f (127.9.236.3:37265), f6cf7d4003f44ed79106c606f32d44c6 (127.9.236.1:44225)
I20260929 19:14:04.103840 28698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.100250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:60443 (local address 127.9.236.3:37265)
0929 19:14:04.100306 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:04.100308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:04.100321 (+ 13us) server_negotiation.cc:407] Connection header received
0929 19:14:04.100352 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:04.100353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:04.100364 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:04.100424 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:04.100670 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.101419 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.102873 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.103469 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.103539 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.103552 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:04.103718 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 19:14:04.103760 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 19:14:04.103840 28697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.100448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:54039 (local address 127.9.236.1:44225)
0929 19:14:04.100497 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:04.100498 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:04.100506 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:14:04.100571 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:04.100572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:04.100581 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:04.100645 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:04.100904 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.101595 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.102872 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.103460 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.103543 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.103552 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:04.103718 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 19:14:04.103760 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260929 19:14:04.103839 28550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.100191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:37265 (local address 127.9.236.2:60443)
0929 19:14:04.100236 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:04.100249 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:04.100295 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:04.100478 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:04.100479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:04.100482 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:04.100585 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.100588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.101495 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.101496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.102767 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.102770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.103546 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.103547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.103640 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.103642 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:14:04.103701 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 19:14:04.103759 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260929 19:14:04.103844 28708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.100298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:44225 (local address 127.9.236.2:54039)
0929 19:14:04.100459 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:04.100467 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:04.100508 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:04.100735 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:04.100737 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:04.100740 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:04.100825 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.100827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.101666 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.101690 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.102767 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.102770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.103549 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.103550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.103637 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.103642 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:14:04.103701 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 19:14:04.103742 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":1,"thread_start_us":118,"threads_started":1}
I20260929 19:14:04.104362 28368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be797b99d7664887ac07a0d1267f975e" candidate_uuid: "f33fb5f0f8394f23948b2127da5a2777" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6cf7d4003f44ed79106c606f32d44c6" is_pre_election: true
I20260929 19:14:04.104384 28636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be797b99d7664887ac07a0d1267f975e" candidate_uuid: "f33fb5f0f8394f23948b2127da5a2777" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "328f0a45507b4cd1a079c4a4e928e71f" is_pre_election: true
I20260929 19:14:04.104519 28368 raft_consensus.cc:2454] T be797b99d7664887ac07a0d1267f975e P f6cf7d4003f44ed79106c606f32d44c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f33fb5f0f8394f23948b2127da5a2777 in term 0.
I20260929 19:14:04.104580 28636 raft_consensus.cc:2454] T be797b99d7664887ac07a0d1267f975e P 328f0a45507b4cd1a079c4a4e928e71f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f33fb5f0f8394f23948b2127da5a2777 in term 0.
I20260929 19:14:04.104741 28437 leader_election.cc:302] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f33fb5f0f8394f23948b2127da5a2777, f6cf7d4003f44ed79106c606f32d44c6; no voters:
I20260929 19:14:04.104889 28707 raft_consensus.cc:2781] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:14:04.104969 28707 raft_consensus.cc:482] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:14:04.104985 28707 raft_consensus.cc:3037] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:04.105876 28707 raft_consensus.cc:504] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "328f0a45507b4cd1a079c4a4e928e71f" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 37265 } } peers { permanent_uuid: "f33fb5f0f8394f23948b2127da5a2777" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 33723 } } peers { permanent_uuid: "f6cf7d4003f44ed79106c606f32d44c6" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44225 } }
I20260929 19:14:04.106019 28707 leader_election.cc:288] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 [CANDIDATE]: Term 1 election: Requested vote from peers 328f0a45507b4cd1a079c4a4e928e71f (127.9.236.3:37265), f6cf7d4003f44ed79106c606f32d44c6 (127.9.236.1:44225)
I20260929 19:14:04.106196 28636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be797b99d7664887ac07a0d1267f975e" candidate_uuid: "f33fb5f0f8394f23948b2127da5a2777" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "328f0a45507b4cd1a079c4a4e928e71f"
I20260929 19:14:04.106220 28368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be797b99d7664887ac07a0d1267f975e" candidate_uuid: "f33fb5f0f8394f23948b2127da5a2777" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6cf7d4003f44ed79106c606f32d44c6"
I20260929 19:14:04.106282 28636 raft_consensus.cc:3037] T be797b99d7664887ac07a0d1267f975e P 328f0a45507b4cd1a079c4a4e928e71f [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:04.106299 28368 raft_consensus.cc:3037] T be797b99d7664887ac07a0d1267f975e P f6cf7d4003f44ed79106c606f32d44c6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:04.107206 28368 raft_consensus.cc:2454] T be797b99d7664887ac07a0d1267f975e P f6cf7d4003f44ed79106c606f32d44c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f33fb5f0f8394f23948b2127da5a2777 in term 1.
I20260929 19:14:04.107210 28636 raft_consensus.cc:2454] T be797b99d7664887ac07a0d1267f975e P 328f0a45507b4cd1a079c4a4e928e71f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f33fb5f0f8394f23948b2127da5a2777 in term 1.
I20260929 19:14:04.107362 28437 leader_election.cc:302] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 328f0a45507b4cd1a079c4a4e928e71f, f33fb5f0f8394f23948b2127da5a2777; no voters:
I20260929 19:14:04.107501 28707 raft_consensus.cc:2781] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:04.107702 28707 raft_consensus.cc:686] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 [term 1 LEADER]: Becoming Leader. State: Replica: f33fb5f0f8394f23948b2127da5a2777, State: Running, Role: LEADER
I20260929 19:14:04.107815 28707 consensus_queue.cc:237] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "328f0a45507b4cd1a079c4a4e928e71f" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 37265 } } peers { permanent_uuid: "f33fb5f0f8394f23948b2127da5a2777" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 33723 } } peers { permanent_uuid: "f6cf7d4003f44ed79106c606f32d44c6" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44225 } }
I20260929 19:14:04.108860 28222 catalog_manager.cc:5495] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 reported cstate change: term changed from 0 to 1, leader changed from <none> to f33fb5f0f8394f23948b2127da5a2777 (127.9.236.2). New cstate: current_term: 1 leader_uuid: "f33fb5f0f8394f23948b2127da5a2777" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "328f0a45507b4cd1a079c4a4e928e71f" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 37265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f33fb5f0f8394f23948b2127da5a2777" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 33723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6cf7d4003f44ed79106c606f32d44c6" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44225 } health_report { overall_health: UNKNOWN } } }
W20260929 19:14:04.120388 28549 tablet.cc:2369] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:14:04.121019 28695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.118620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36764 (local address 127.9.236.2:33723)
0929 19:14:04.118683 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:04.118685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:04.118694 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:14:04.118770 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:04.118771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:04.118795 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:04.118855 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:04.119002 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.119500 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.120248 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.120450 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.120504 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.120659 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:04.120756 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:04.120840 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:04.120901 (+ 61us) server_negotiation.cc:299] Negotiation successful
0929 19:14:04.120910 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 19:14:04.122843 28368 raft_consensus.cc:1261] T be797b99d7664887ac07a0d1267f975e P f6cf7d4003f44ed79106c606f32d44c6 [term 1 FOLLOWER]: Refusing update from remote peer f33fb5f0f8394f23948b2127da5a2777: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:14:04.122906 28636 raft_consensus.cc:1261] T be797b99d7664887ac07a0d1267f975e P 328f0a45507b4cd1a079c4a4e928e71f [term 1 FOLLOWER]: Refusing update from remote peer f33fb5f0f8394f23948b2127da5a2777: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:14:04.123168 28707 consensus_queue.cc:1035] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6cf7d4003f44ed79106c606f32d44c6" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 44225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:14:04.123312 28709 consensus_queue.cc:1035] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 [LEADER]: Connected to new peer: Peer: permanent_uuid: "328f0a45507b4cd1a079c4a4e928e71f" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 37265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:14:04.126300 28714 mvcc.cc:205] Tried to move back new op lower bound from 7334745063924629504 to 7334745063866437632. Current Snapshot: MvccSnapshot[applied={T|T < 7334745063924629504}]
I20260929 19:14:04.128504 28222 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:14:04.128618 28222 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:14:04.129477 28222 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a2317f9ac1e4a3cb0a3b86679a58592: Sending DeleteTablet for 3 replicas of tablet be797b99d7664887ac07a0d1267f975e
I20260929 19:14:04.129910 28616 tablet_service.cc:1514] Processing DeleteTablet for tablet be797b99d7664887ac07a0d1267f975e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:14:04 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:36104
I20260929 19:14:04.129921 28348 tablet_service.cc:1514] Processing DeleteTablet for tablet be797b99d7664887ac07a0d1267f975e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:14:04 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:60882
I20260929 19:14:04.130051 28482 tablet_service.cc:1514] Processing DeleteTablet for tablet be797b99d7664887ac07a0d1267f975e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:14:04 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:36762
I20260929 19:14:04.130323 28724 tablet_replica.cc:330] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777: stopping tablet replica
I20260929 19:14:04.130395 28723 tablet_replica.cc:330] T be797b99d7664887ac07a0d1267f975e P 328f0a45507b4cd1a079c4a4e928e71f: stopping tablet replica
I20260929 19:14:04.130468 28724 raft_consensus.cc:2229] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:14:04.130515 28723 raft_consensus.cc:2229] T be797b99d7664887ac07a0d1267f975e P 328f0a45507b4cd1a079c4a4e928e71f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:14:04.130648 28723 raft_consensus.cc:2258] T be797b99d7664887ac07a0d1267f975e P 328f0a45507b4cd1a079c4a4e928e71f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:14:04.130749 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 28285
I20260929 19:14:04.130834 28722 tablet_replica.cc:330] T be797b99d7664887ac07a0d1267f975e P f6cf7d4003f44ed79106c606f32d44c6: stopping tablet replica
I20260929 19:14:04.130923 28722 raft_consensus.cc:2229] T be797b99d7664887ac07a0d1267f975e P f6cf7d4003f44ed79106c606f32d44c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:14:04.131001 28722 raft_consensus.cc:2258] T be797b99d7664887ac07a0d1267f975e P f6cf7d4003f44ed79106c606f32d44c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:14:04.132412 28723 ts_tablet_manager.cc:1905] T be797b99d7664887ac07a0d1267f975e P 328f0a45507b4cd1a079c4a4e928e71f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:14:04.133229 28724 raft_consensus.cc:2258] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:14:04.133665 28724 ts_tablet_manager.cc:1905] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:14:04.134698 28723 ts_tablet_manager.cc:1918] T be797b99d7664887ac07a0d1267f975e P 328f0a45507b4cd1a079c4a4e928e71f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:14:04.134822 28723 log.cc:1196] T be797b99d7664887ac07a0d1267f975e P 328f0a45507b4cd1a079c4a4e928e71f: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/be797b99d7664887ac07a0d1267f975e
I20260929 19:14:04.135155 28723 ts_tablet_manager.cc:1939] T be797b99d7664887ac07a0d1267f975e P 328f0a45507b4cd1a079c4a4e928e71f: Deleting consensus metadata
I20260929 19:14:04.135398 28724 ts_tablet_manager.cc:1918] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:14:04.135464 28724 log.cc:1196] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/be797b99d7664887ac07a0d1267f975e
I20260929 19:14:04.135706 28724 ts_tablet_manager.cc:1939] T be797b99d7664887ac07a0d1267f975e P f33fb5f0f8394f23948b2127da5a2777: Deleting consensus metadata
I20260929 19:14:04.135784 28209 catalog_manager.cc:4841] TS 328f0a45507b4cd1a079c4a4e928e71f (127.9.236.3:37265): tablet be797b99d7664887ac07a0d1267f975e (table test-table [id=71d817f684f04e8e8522a17232454816]) successfully deleted
I20260929 19:14:04.136085 28210 catalog_manager.cc:4841] TS f33fb5f0f8394f23948b2127da5a2777 (127.9.236.2:33723): tablet be797b99d7664887ac07a0d1267f975e (table test-table [id=71d817f684f04e8e8522a17232454816]) successfully deleted
W20260929 19:14:04.136633 28209 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.236.1:44225 (error 108)
W20260929 19:14:04.137208 28694 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:04.137039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:44225 (local address 127.0.0.1:60888)
0929 19:14:04.137123 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:04.137164 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.1:44225: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 19:14:04.137331 28209 catalog_manager.cc:4584] TS f6cf7d4003f44ed79106c606f32d44c6 (127.9.236.1:44225): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet be797b99d7664887ac07a0d1267f975e: Network error: Client connection negotiation failed: client connection to 127.9.236.1:44225: connect: Connection reset by peer (error 104)
I20260929 19:14:04.137717 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 28419
I20260929 19:14:04.142485 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 28553
I20260929 19:14:04.146641 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 28191
2026-09-29T19:14:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1816 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-29T19:14:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:04Z Disabled control of system clock
I20260929 19:14:04.175096 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:33655
--webserver_interface=127.9.236.62
--webserver_port=0
--builtin_ntp_servers=127.9.236.20:42511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:33655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 19:14:04.261472 28729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:04.261742 28729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:04.261763 28729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:04.262945 28729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:04.262984 28729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:04.262995 28729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:04.263006 28729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:04.263016 28729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:04.264402 28729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:42511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:33655
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:33655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:04.264637 28729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:04.264914 28729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:04.269922 28737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:04.269932 28735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:04.269955 28734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:04.270263 28729 server_base.cc:1034] running on GCE node
I20260929 19:14:04.270550 28729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:04.270840 28729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:04.271996 28729 hybrid_clock.cc:648] HybridClock initialized: now 1790709244271973 us; error 35 us; skew 500 ppm
I20260929 19:14:04.273288 28729 webserver.cc:460] Webserver started at http://127.9.236.62:46735/ using document root <none> and password file <none>
I20260929 19:14:04.273527 28729 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:04.273592 28729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:04.273716 28729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:04.274760 28729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "1f85502be0424992b83f96db5aa3967b"
format_stamp: "Formatted at 2026-09-29 19:14:04 on dist-test-slave-wftq"
server_key: "3ddb39ed92026afca5f1dec9568caf12"
server_key_iv: "391b74399ba96c6465709ac0f994c39e"
server_key_version: "encryptionkey@0"
I20260929 19:14:04.275059 28729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "1f85502be0424992b83f96db5aa3967b"
format_stamp: "Formatted at 2026-09-29 19:14:04 on dist-test-slave-wftq"
server_key: "3ddb39ed92026afca5f1dec9568caf12"
server_key_iv: "391b74399ba96c6465709ac0f994c39e"
server_key_version: "encryptionkey@0"
I20260929 19:14:04.276376 28729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:14:04.277184 28743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:04.277401 28729 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:04.277482 28729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "1f85502be0424992b83f96db5aa3967b"
format_stamp: "Formatted at 2026-09-29 19:14:04 on dist-test-slave-wftq"
server_key: "3ddb39ed92026afca5f1dec9568caf12"
server_key_iv: "391b74399ba96c6465709ac0f994c39e"
server_key_version: "encryptionkey@0"
I20260929 19:14:04.277537 28729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:04.286063 28729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:04.286324 28729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:04.286427 28729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:04.290472 28729 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:33655
I20260929 19:14:04.290541 28795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:33655 every 8 connection(s)
I20260929 19:14:04.290851 28729 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:04.291967 28796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:04.294607 28796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b: Bootstrap starting.
I20260929 19:14:04.295239 28796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:04.295531 28796 log.cc:824] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:04.296249 28796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b: No bootstrap required, opened a new log
I20260929 19:14:04.298044 28796 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f85502be0424992b83f96db5aa3967b" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 33655 } }
I20260929 19:14:04.298193 28796 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:04.298215 28796 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f85502be0424992b83f96db5aa3967b, State: Initialized, Role: FOLLOWER
I20260929 19:14:04.298362 28796 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f85502be0424992b83f96db5aa3967b" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 33655 } }
I20260929 19:14:04.298439 28796 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:04.298461 28796 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:04.298491 28796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:04.299016 28796 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f85502be0424992b83f96db5aa3967b" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 33655 } }
I20260929 19:14:04.299109 28796 leader_election.cc:302] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f85502be0424992b83f96db5aa3967b; no voters:
I20260929 19:14:04.299284 28796 leader_election.cc:288] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:04.299355 28799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:04.299580 28796 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:04.299592 28799 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b [term 1 LEADER]: Becoming Leader. State: Replica: 1f85502be0424992b83f96db5aa3967b, State: Running, Role: LEADER
I20260929 19:14:04.299741 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 28729
I20260929 19:14:04.299852 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:04.299796 28799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f85502be0424992b83f96db5aa3967b" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 33655 } }
I20260929 19:14:04.300076 10160 external_mini_cluster.cc:1428] Setting key 17f113c7b82840d68fdbf4e37ca68538
I20260929 19:14:04.300201 28801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f85502be0424992b83f96db5aa3967b. Latest consensus state: current_term: 1 leader_uuid: "1f85502be0424992b83f96db5aa3967b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f85502be0424992b83f96db5aa3967b" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 33655 } } }
I20260929 19:14:04.300279 28801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:04.300289 28800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f85502be0424992b83f96db5aa3967b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f85502be0424992b83f96db5aa3967b" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 33655 } } }
I20260929 19:14:04.300367 28800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:04.301122 28804 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:04.301779 28804 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:04.303452 28804 catalog_manager.cc:1347] Generated new cluster ID: b48c292ee27443558580280dba162fae
I20260929 19:14:04.303508 28804 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:04.306784 28811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.300814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41228 (local address 127.9.236.62:33655)
0929 19:14:04.301023 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:04.301034 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:04.301072 (+ 38us) server_negotiation.cc:407] Connection header received
0929 19:14:04.301242 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:04.301250 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:04.301341 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:04.301507 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:04.301764 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.303185 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.304282 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.304925 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.305057 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.305804 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:04.305808 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:04.305809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:04.306061 (+ 252us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:04.306109 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:04.306142 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:04.306218 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:04.306290 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 19:14:04.306530 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":74,"threads_started":1}
I20260929 19:14:04.316257 28804 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:04.316828 28804 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:04.319877 28804 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b: Generated new TSK 0
I20260929 19:14:04.320041 28804 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:04.326074 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:33655
--builtin_ntp_servers=127.9.236.20:42511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 19:14:04.431311 28820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:04.431581 28820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:04.431615 28820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:04.432899 28820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:04.432940 28820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:04.432974 28820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:14:04.434397 28820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:42511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:33655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:04.434679 28820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:04.434988 28820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:04.440402 28825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:04.440424 28826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:04.440591 28828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:04.440819 28820 server_base.cc:1034] running on GCE node
I20260929 19:14:04.441032 28820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:04.441293 28820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:04.442453 28820 hybrid_clock.cc:648] HybridClock initialized: now 1790709244442435 us; error 35 us; skew 500 ppm
I20260929 19:14:04.443795 28820 webserver.cc:460] Webserver started at http://127.9.236.1:45349/ using document root <none> and password file <none>
I20260929 19:14:04.444046 28820 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:04.444095 28820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:04.444227 28820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:04.445451 28820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "10ab06ff221f433890e068f5ba0a9304"
format_stamp: "Formatted at 2026-09-29 19:14:04 on dist-test-slave-wftq"
server_key: "468d9244b60a893456001c36eeefb1b3"
server_key_iv: "44702358e966a4c2f164881dbc786456"
server_key_version: "encryptionkey@0"
I20260929 19:14:04.445796 28820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "10ab06ff221f433890e068f5ba0a9304"
format_stamp: "Formatted at 2026-09-29 19:14:04 on dist-test-slave-wftq"
server_key: "468d9244b60a893456001c36eeefb1b3"
server_key_iv: "44702358e966a4c2f164881dbc786456"
server_key_version: "encryptionkey@0"
I20260929 19:14:04.447068 28820 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:14:04.447785 28834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:04.447961 28820 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:14:04.448032 28820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "10ab06ff221f433890e068f5ba0a9304"
format_stamp: "Formatted at 2026-09-29 19:14:04 on dist-test-slave-wftq"
server_key: "468d9244b60a893456001c36eeefb1b3"
server_key_iv: "44702358e966a4c2f164881dbc786456"
server_key_version: "encryptionkey@0"
I20260929 19:14:04.448104 28820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:04.468560 28820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:04.468921 28820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:04.469058 28820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:04.469367 28820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:14:04.469748 28820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:14:04.469784 28820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:04.469810 28820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:14:04.469834 28820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:04.476941 28820 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:40625
I20260929 19:14:04.477010 28947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:40625 every 8 connection(s)
I20260929 19:14:04.477396 28820 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:14:04.482182 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 28820
I20260929 19:14:04.482288 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:14:04.482518 10160 external_mini_cluster.cc:1428] Setting key 6ca7b86e9c20a31e7c2a361cc4c59b99
I20260929 19:14:04.483006 28811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.478231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:43337 (local address 127.9.236.62:33655)
0929 19:14:04.478306 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:04.478308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:04.478568 (+ 260us) server_negotiation.cc:407] Connection header received
0929 19:14:04.479136 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:04.479137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:04.479153 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:04.479198 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:04.479915 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.480486 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.481527 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.481730 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.481783 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.482650 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:04.482652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:04.482653 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:04.482659 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:04.482684 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:04.482691 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:04.482770 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:04.482883 (+ 113us) server_negotiation.cc:299] Negotiation successful
0929 19:14:04.482938 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 19:14:04.483188 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.478202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:33655 (local address 127.9.236.1:43337)
0929 19:14:04.478432 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:04.478469 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:04.479025 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:04.479271 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:04.479274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:04.479282 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:04.479832 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.479836 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.480569 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.480570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.481450 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.481452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.481799 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.481799 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.481919 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.482497 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:14:04.482498 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:04.482527 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:14:04.482549 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:04.482791 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:04.482796 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:04.482799 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:14:04.482875 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 19:14:04.482976 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":108,"threads_started":1}
I20260929 19:14:04.483745 28948 heartbeater.cc:347] Connected to a master server at 127.9.236.62:33655
I20260929 19:14:04.483759 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:33655
--builtin_ntp_servers=127.9.236.20:42511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 19:14:04.483855 28948 heartbeater.cc:464] Registering TS with master...
I20260929 19:14:04.484040 28948 heartbeater.cc:511] Master 127.9.236.62:33655 requested a full tablet report, sending...
I20260929 19:14:04.484594 28760 ts_manager.cc:194] Registered new tserver with Master: 10ab06ff221f433890e068f5ba0a9304 (127.9.236.1:40625)
I20260929 19:14:04.485320 28760 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.236.1:43337
W20260929 19:14:04.566270 28951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:04.566483 28951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:04.566504 28951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:04.567656 28951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:04.567709 28951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:04.567734 28951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:14:04.569046 28951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:42511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:33655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:04.569296 28951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:04.569576 28951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:04.574657 28957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:04.574674 28956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:04.574705 28959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:04.574869 28951 server_base.cc:1034] running on GCE node
I20260929 19:14:04.575083 28951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:04.575312 28951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:04.576450 28951 hybrid_clock.cc:648] HybridClock initialized: now 1790709244576446 us; error 36 us; skew 500 ppm
I20260929 19:14:04.577776 28951 webserver.cc:460] Webserver started at http://127.9.236.2:35021/ using document root <none> and password file <none>
I20260929 19:14:04.577993 28951 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:04.578042 28951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:04.578171 28951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:04.579217 28951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "6ef6c234e33c41c0b428727d393b9d1c"
format_stamp: "Formatted at 2026-09-29 19:14:04 on dist-test-slave-wftq"
server_key: "0a0f1da7ea74d477e0937970523fae37"
server_key_iv: "1a6961e3bb9bb2502bda75bcdbb40fa6"
server_key_version: "encryptionkey@0"
I20260929 19:14:04.579541 28951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "6ef6c234e33c41c0b428727d393b9d1c"
format_stamp: "Formatted at 2026-09-29 19:14:04 on dist-test-slave-wftq"
server_key: "0a0f1da7ea74d477e0937970523fae37"
server_key_iv: "1a6961e3bb9bb2502bda75bcdbb40fa6"
server_key_version: "encryptionkey@0"
I20260929 19:14:04.580832 28951 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:14:04.581728 28965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:04.581925 28951 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 19:14:04.582012 28951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "6ef6c234e33c41c0b428727d393b9d1c"
format_stamp: "Formatted at 2026-09-29 19:14:04 on dist-test-slave-wftq"
server_key: "0a0f1da7ea74d477e0937970523fae37"
server_key_iv: "1a6961e3bb9bb2502bda75bcdbb40fa6"
server_key_version: "encryptionkey@0"
I20260929 19:14:04.582082 28951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:04.594869 28951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:04.595144 28951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:04.595284 28951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:04.595491 28951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:14:04.595805 28951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:14:04.595841 28951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:04.595868 28951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:14:04.595884 28951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:04.602157 28951 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:45249
I20260929 19:14:04.602226 29078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:45249 every 8 connection(s)
I20260929 19:14:04.602512 28951 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:14:04.607962 28811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.603350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:37865 (local address 127.9.236.62:33655)
0929 19:14:04.603447 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:04.603449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:04.603609 (+ 160us) server_negotiation.cc:407] Connection header received
0929 19:14:04.604118 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:04.604119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:04.604133 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:04.604236 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:04.604878 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.605509 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.606480 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.606690 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.606760 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.607583 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:04.607584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:04.607584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:04.607589 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:04.607605 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:04.607609 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:04.607670 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:04.607811 (+ 141us) server_negotiation.cc:299] Negotiation successful
0929 19:14:04.607842 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 19:14:04.608004 29081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.603282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:33655 (local address 127.9.236.2:37865)
0929 19:14:04.603474 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:04.603514 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:04.604011 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:04.604298 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:04.604302 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:04.604309 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:04.604782 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.604785 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.605605 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.605606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.606391 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.606393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.606770 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.606771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.606876 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.607421 (+ 545us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:14:04.607421 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:04.607446 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:14:04.607468 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:04.607698 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:04.607703 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:04.607707 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:14:04.607770 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 19:14:04.607848 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260929 19:14:04.608673 29079 heartbeater.cc:347] Connected to a master server at 127.9.236.62:33655
I20260929 19:14:04.608771 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 28951
I20260929 19:14:04.608820 29079 heartbeater.cc:464] Registering TS with master...
I20260929 19:14:04.608884 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:14:04.609136 29079 heartbeater.cc:511] Master 127.9.236.62:33655 requested a full tablet report, sending...
I20260929 19:14:04.609189 10160 external_mini_cluster.cc:1428] Setting key 2025378dc05efe5dcab9535a7815841d
I20260929 19:14:04.609611 28760 ts_manager.cc:194] Registered new tserver with Master: 6ef6c234e33c41c0b428727d393b9d1c (127.9.236.2:45249)
I20260929 19:14:04.610133 28760 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.236.2:37865
I20260929 19:14:04.610441 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:33655
--builtin_ntp_servers=127.9.236.20:42511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 19:14:04.701313 29082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:04.701546 29082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:04.701573 29082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:04.702862 29082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:04.702903 29082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:04.702929 29082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:14:04.704257 29082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:42511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:33655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:04.704469 29082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:04.704752 29082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:04.710078 29090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:04.710036 29088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:04.710041 29087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:04.710484 29082 server_base.cc:1034] running on GCE node
I20260929 19:14:04.710713 29082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:04.710971 29082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:04.712128 29082 hybrid_clock.cc:648] HybridClock initialized: now 1790709244712109 us; error 36 us; skew 500 ppm
I20260929 19:14:04.713408 29082 webserver.cc:460] Webserver started at http://127.9.236.3:40929/ using document root <none> and password file <none>
I20260929 19:14:04.713692 29082 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:04.713745 29082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:04.713867 29082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:04.714911 29082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "716c990e2a734a7d86699693fe055b4b"
format_stamp: "Formatted at 2026-09-29 19:14:04 on dist-test-slave-wftq"
server_key: "46c1433ced1ec0eb5b50b76a92033346"
server_key_iv: "1c10f9d2faddcf2b66de097bd4d263b7"
server_key_version: "encryptionkey@0"
I20260929 19:14:04.715238 29082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "716c990e2a734a7d86699693fe055b4b"
format_stamp: "Formatted at 2026-09-29 19:14:04 on dist-test-slave-wftq"
server_key: "46c1433ced1ec0eb5b50b76a92033346"
server_key_iv: "1c10f9d2faddcf2b66de097bd4d263b7"
server_key_version: "encryptionkey@0"
I20260929 19:14:04.716506 29082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:14:04.717216 29096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:04.717406 29082 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:14:04.717491 29082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "716c990e2a734a7d86699693fe055b4b"
format_stamp: "Formatted at 2026-09-29 19:14:04 on dist-test-slave-wftq"
server_key: "46c1433ced1ec0eb5b50b76a92033346"
server_key_iv: "1c10f9d2faddcf2b66de097bd4d263b7"
server_key_version: "encryptionkey@0"
I20260929 19:14:04.717553 29082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:04.730808 29082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:04.731055 29082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:04.731176 29082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:04.731379 29082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:14:04.731681 29082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:14:04.731714 29082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:04.731741 29082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:14:04.731758 29082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:04.737898 29082 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:42897
I20260929 19:14:04.737957 29209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:42897 every 8 connection(s)
I20260929 19:14:04.738245 29082 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:14:04.743662 28811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.739048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:55971 (local address 127.9.236.62:33655)
0929 19:14:04.739103 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:04.739105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:04.739311 (+ 206us) server_negotiation.cc:407] Connection header received
0929 19:14:04.739887 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:04.739888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:04.739898 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:04.739955 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:04.740704 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.741260 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.742296 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.742506 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.742574 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.743415 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:04.743417 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:04.743417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:04.743422 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:04.743438 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:04.743442 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:04.743488 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:04.743562 (+ 74us) server_negotiation.cc:299] Negotiation successful
0929 19:14:04.743585 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 19:14:04.743892 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.738953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:33655 (local address 127.9.236.3:55971)
0929 19:14:04.739158 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:04.739200 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:04.739800 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:04.740036 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:04.740040 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:04.740049 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:04.740610 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.740614 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.741342 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.741342 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.742200 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.742202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.742590 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.742591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.742708 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.743263 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:14:04.743263 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:04.743294 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:14:04.743320 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:04.743497 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:04.743500 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:04.743502 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:14:04.743554 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 19:14:04.743655 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260929 19:14:04.744465 29210 heartbeater.cc:347] Connected to a master server at 127.9.236.62:33655
I20260929 19:14:04.744563 29210 heartbeater.cc:464] Registering TS with master...
I20260929 19:14:04.744758 29210 heartbeater.cc:511] Master 127.9.236.62:33655 requested a full tablet report, sending...
I20260929 19:14:04.745240 28760 ts_manager.cc:194] Registered new tserver with Master: 716c990e2a734a7d86699693fe055b4b (127.9.236.3:42897)
I20260929 19:14:04.745609 28760 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.236.3:55971
I20260929 19:14:04.746289 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 29082
I20260929 19:14:04.746409 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:14:04.746616 10160 external_mini_cluster.cc:1428] Setting key 6ceb6916c734eac1717a9d40b829196c
I20260929 19:14:04.747488 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:14:04.752127 28811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.749032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44546 (local address 127.9.236.62:33655)
0929 19:14:04.749131 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:04.749133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:04.749238 (+ 105us) server_negotiation.cc:407] Connection header received
0929 19:14:04.749308 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:04.749309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:04.749324 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:04.749372 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:04.749532 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.750063 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.750886 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.751085 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.751140 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.751817 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:04.751818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:04.751819 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:04.751823 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:04.751838 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:04.751843 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:04.751907 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:04.752022 (+ 115us) server_negotiation.cc:299] Negotiation successful
0929 19:14:04.752034 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 19:14:04.753875 28760 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44546:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:14:04.755748 28760 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:14:04.762523 29223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.758624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38798 (local address 127.9.236.3:42897)
0929 19:14:04.758804 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:04.758809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:04.758914 (+ 105us) server_negotiation.cc:407] Connection header received
0929 19:14:04.759018 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:04.759020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:04.759034 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:04.759079 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:04.759250 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.760346 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.761546 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.762089 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.762168 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.762273 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:04.762370 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 19:14:04.762432 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":87,"threads_started":1}
I20260929 19:14:04.762506 29218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.758426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:42897 (local address 127.0.0.1:38798)
0929 19:14:04.758858 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:04.758867 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:04.758938 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:04.759152 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:04.759153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:04.759156 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:04.759186 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.759187 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.760432 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.760432 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.761461 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.761464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.762183 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.762184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.762268 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.762282 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 19:14:04.762351 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 19:14:04.762422 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":73,"threads_started":1}
I20260929 19:14:04.762687 29220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.758342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37786 (local address 127.9.236.1:40625)
0929 19:14:04.758649 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:04.758653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:04.758665 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:14:04.758801 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:04.758803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:04.758821 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:04.758892 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:04.759151 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.759883 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.761328 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.762105 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.762181 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.762267 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:04.762548 (+ 281us) server_negotiation.cc:299] Negotiation successful
0929 19:14:04.762591 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":111,"threads_started":1}
I20260929 19:14:04.762792 29219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.758230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:40625 (local address 127.0.0.1:37786)
0929 19:14:04.758564 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:04.758601 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:04.758677 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:04.758984 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:04.758987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:04.758990 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:04.759074 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.759077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.759992 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.759993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.761205 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.761208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.762319 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.762320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.762388 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.762393 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 19:14:04.762437 (+ 44us) client_negotiation.cc:240] Negotiation successful
0929 19:14:04.762497 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":139,"threads_started":1}
I20260929 19:14:04.763469 29144 tablet_service.cc:1467] Processing CreateTablet for tablet 06aa0e926bd7499d86f41acff7496132 (DEFAULT_TABLE table=test-table [id=e46d32bcd9b64606a65c631fc5c047bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:14:04.763604 28882 tablet_service.cc:1467] Processing CreateTablet for tablet 06aa0e926bd7499d86f41acff7496132 (DEFAULT_TABLE table=test-table [id=e46d32bcd9b64606a65c631fc5c047bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:14:04.763855 29144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06aa0e926bd7499d86f41acff7496132. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:04.763993 28882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06aa0e926bd7499d86f41acff7496132. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:04.764413 29222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.758188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:45249 (local address 127.0.0.1:55040)
0929 19:14:04.758560 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:04.758602 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:04.758681 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:04.759036 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:04.759037 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:04.759041 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:04.759102 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.759105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.761159 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.761159 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.762301 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.762304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.764185 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.764186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.764244 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.764247 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:14:04.764292 (+ 45us) client_negotiation.cc:240] Negotiation successful
0929 19:14:04.764336 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":71,"threads_started":1}
I20260929 19:14:04.764693 29221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.758342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55040 (local address 127.9.236.2:45249)
0929 19:14:04.758532 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:04.758538 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:04.758799 (+ 261us) server_negotiation.cc:407] Connection header received
0929 19:14:04.758862 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:04.758866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:04.758885 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:04.758957 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:04.759175 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.760333 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.762387 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.764006 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.764072 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.764150 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:04.764576 (+ 426us) server_negotiation.cc:299] Negotiation successful
0929 19:14:04.764617 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":114,"threads_started":1}
I20260929 19:14:04.765408 29013 tablet_service.cc:1467] Processing CreateTablet for tablet 06aa0e926bd7499d86f41acff7496132 (DEFAULT_TABLE table=test-table [id=e46d32bcd9b64606a65c631fc5c047bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:14:04.765759 29013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06aa0e926bd7499d86f41acff7496132. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:04.766934 29224 tablet_bootstrap.cc:492] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b: Bootstrap starting.
I20260929 19:14:04.767565 29224 tablet_bootstrap.cc:654] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:04.767580 29225 tablet_bootstrap.cc:492] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304: Bootstrap starting.
I20260929 19:14:04.767998 29224 log.cc:824] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:04.768218 29225 tablet_bootstrap.cc:654] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:04.768525 29225 log.cc:824] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:04.768939 29224 tablet_bootstrap.cc:492] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b: No bootstrap required, opened a new log
I20260929 19:14:04.769030 29224 ts_tablet_manager.cc:1397] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:14:04.769253 29225 tablet_bootstrap.cc:492] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304: No bootstrap required, opened a new log
I20260929 19:14:04.769253 29228 tablet_bootstrap.cc:492] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c: Bootstrap starting.
I20260929 19:14:04.769339 29225 ts_tablet_manager.cc:1397] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 19:14:04.770005 29228 tablet_bootstrap.cc:654] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:04.770364 29228 log.cc:824] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:04.771377 29228 tablet_bootstrap.cc:492] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c: No bootstrap required, opened a new log
I20260929 19:14:04.771252 29224 raft_consensus.cc:348] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef6c234e33c41c0b428727d393b9d1c" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45249 } } peers { permanent_uuid: "716c990e2a734a7d86699693fe055b4b" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 42897 } } peers { permanent_uuid: "10ab06ff221f433890e068f5ba0a9304" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 40625 } }
I20260929 19:14:04.771430 29224 raft_consensus.cc:374] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:04.771478 29224 raft_consensus.cc:729] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 716c990e2a734a7d86699693fe055b4b, State: Initialized, Role: FOLLOWER
I20260929 19:14:04.771587 29228 ts_tablet_manager.cc:1397] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:14:04.771624 29224 consensus_queue.cc:260] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef6c234e33c41c0b428727d393b9d1c" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45249 } } peers { permanent_uuid: "716c990e2a734a7d86699693fe055b4b" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 42897 } } peers { permanent_uuid: "10ab06ff221f433890e068f5ba0a9304" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 40625 } }
I20260929 19:14:04.771924 29224 ts_tablet_manager.cc:1428] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:14:04.771979 29210 heartbeater.cc:503] Master 127.9.236.62:33655 was elected leader, sending a full tablet report...
I20260929 19:14:04.771898 29225 raft_consensus.cc:348] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef6c234e33c41c0b428727d393b9d1c" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45249 } } peers { permanent_uuid: "716c990e2a734a7d86699693fe055b4b" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 42897 } } peers { permanent_uuid: "10ab06ff221f433890e068f5ba0a9304" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 40625 } }
I20260929 19:14:04.772063 29225 raft_consensus.cc:374] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:04.772104 29225 raft_consensus.cc:729] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10ab06ff221f433890e068f5ba0a9304, State: Initialized, Role: FOLLOWER
I20260929 19:14:04.772387 29225 consensus_queue.cc:260] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef6c234e33c41c0b428727d393b9d1c" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45249 } } peers { permanent_uuid: "716c990e2a734a7d86699693fe055b4b" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 42897 } } peers { permanent_uuid: "10ab06ff221f433890e068f5ba0a9304" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 40625 } }
I20260929 19:14:04.772692 29225 ts_tablet_manager.cc:1428] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 19:14:04.772753 28948 heartbeater.cc:503] Master 127.9.236.62:33655 was elected leader, sending a full tablet report...
I20260929 19:14:04.774406 29228 raft_consensus.cc:348] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef6c234e33c41c0b428727d393b9d1c" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45249 } } peers { permanent_uuid: "716c990e2a734a7d86699693fe055b4b" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 42897 } } peers { permanent_uuid: "10ab06ff221f433890e068f5ba0a9304" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 40625 } }
I20260929 19:14:04.774531 29228 raft_consensus.cc:374] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:04.774559 29228 raft_consensus.cc:729] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ef6c234e33c41c0b428727d393b9d1c, State: Initialized, Role: FOLLOWER
I20260929 19:14:04.774690 29228 consensus_queue.cc:260] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef6c234e33c41c0b428727d393b9d1c" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45249 } } peers { permanent_uuid: "716c990e2a734a7d86699693fe055b4b" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 42897 } } peers { permanent_uuid: "10ab06ff221f433890e068f5ba0a9304" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 40625 } }
I20260929 19:14:04.774920 29228 ts_tablet_manager.cc:1428] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 19:14:04.774993 29079 heartbeater.cc:503] Master 127.9.236.62:33655 was elected leader, sending a full tablet report...
W20260929 19:14:04.853119 29080 tablet.cc:2369] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 19:14:04.978145 28949 tablet.cc:2369] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 19:14:04.988829 29211 tablet.cc:2369] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:14:04.992157 29231 raft_consensus.cc:482] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:14:04.992316 29231 raft_consensus.cc:504] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef6c234e33c41c0b428727d393b9d1c" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45249 } } peers { permanent_uuid: "716c990e2a734a7d86699693fe055b4b" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 42897 } } peers { permanent_uuid: "10ab06ff221f433890e068f5ba0a9304" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 40625 } }
I20260929 19:14:04.992769 29231 leader_election.cc:288] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ef6c234e33c41c0b428727d393b9d1c (127.9.236.2:45249), 716c990e2a734a7d86699693fe055b4b (127.9.236.3:42897)
I20260929 19:14:04.995960 29221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.993159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:39255 (local address 127.9.236.2:45249)
0929 19:14:04.993234 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:04.993237 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:04.993315 (+ 78us) server_negotiation.cc:407] Connection header received
0929 19:14:04.993397 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:04.993398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:04.993408 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:04.993461 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:04.993620 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.994098 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.995164 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.995522 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.995596 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.995607 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:04.995802 (+ 195us) server_negotiation.cc:299] Negotiation successful
0929 19:14:04.995871 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 19:14:04.996038 29223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.993370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:59845 (local address 127.9.236.3:42897)
0929 19:14:04.993432 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:04.993434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:04.993447 (+ 13us) server_negotiation.cc:407] Connection header received
0929 19:14:04.993487 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:04.993488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:04.993496 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:04.993543 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:04.993699 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.994209 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.995163 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.995566 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.995630 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.995642 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:04.995900 (+ 258us) server_negotiation.cc:299] Negotiation successful
0929 19:14:04.995980 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 19:14:04.996023 29234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.993243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:42897 (local address 127.9.236.1:59845)
0929 19:14:04.993349 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:04.993362 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:04.993423 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:04.993552 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:04.993553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:04.993555 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:04.993645 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.993648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.994275 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.994276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.995083 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.995085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.995676 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.995677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.995774 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.995782 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 19:14:04.995879 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 19:14:04.995931 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":47,"threads_started":1}
I20260929 19:14:04.996217 29233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:04.993008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:45249 (local address 127.9.236.1:39255)
0929 19:14:04.993219 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:04.993235 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:04.993335 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:04.993474 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:04.993474 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:04.993477 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:04.993559 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.993561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.994182 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.994183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.995072 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:04.995075 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:04.995612 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:04.995613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:04.995698 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:04.995704 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 19:14:04.996056 (+ 352us) client_negotiation.cc:240] Negotiation successful
0929 19:14:04.996123 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":117,"threads_started":1}
I20260929 19:14:04.996482 29164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06aa0e926bd7499d86f41acff7496132" candidate_uuid: "10ab06ff221f433890e068f5ba0a9304" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "716c990e2a734a7d86699693fe055b4b" is_pre_election: true
I20260929 19:14:04.996670 29164 raft_consensus.cc:2454] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10ab06ff221f433890e068f5ba0a9304 in term 0.
I20260929 19:14:04.996622 29033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06aa0e926bd7499d86f41acff7496132" candidate_uuid: "10ab06ff221f433890e068f5ba0a9304" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ef6c234e33c41c0b428727d393b9d1c" is_pre_election: true
I20260929 19:14:04.996784 29033 raft_consensus.cc:2454] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10ab06ff221f433890e068f5ba0a9304 in term 0.
I20260929 19:14:04.996902 28836 leader_election.cc:302] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10ab06ff221f433890e068f5ba0a9304, 716c990e2a734a7d86699693fe055b4b; no voters:
I20260929 19:14:04.997076 29231 raft_consensus.cc:2781] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:14:04.997197 29231 raft_consensus.cc:482] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:14:04.997219 29231 raft_consensus.cc:3037] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:04.998078 29231 raft_consensus.cc:504] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef6c234e33c41c0b428727d393b9d1c" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45249 } } peers { permanent_uuid: "716c990e2a734a7d86699693fe055b4b" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 42897 } } peers { permanent_uuid: "10ab06ff221f433890e068f5ba0a9304" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 40625 } }
I20260929 19:14:04.998215 29231 leader_election.cc:288] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304 [CANDIDATE]: Term 1 election: Requested vote from peers 6ef6c234e33c41c0b428727d393b9d1c (127.9.236.2:45249), 716c990e2a734a7d86699693fe055b4b (127.9.236.3:42897)
I20260929 19:14:04.998471 29033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06aa0e926bd7499d86f41acff7496132" candidate_uuid: "10ab06ff221f433890e068f5ba0a9304" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ef6c234e33c41c0b428727d393b9d1c"
I20260929 19:14:04.998476 29164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06aa0e926bd7499d86f41acff7496132" candidate_uuid: "10ab06ff221f433890e068f5ba0a9304" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "716c990e2a734a7d86699693fe055b4b"
I20260929 19:14:04.998572 29033 raft_consensus.cc:3037] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:04.998576 29164 raft_consensus.cc:3037] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:04.999578 29033 raft_consensus.cc:2454] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10ab06ff221f433890e068f5ba0a9304 in term 1.
I20260929 19:14:04.999578 29164 raft_consensus.cc:2454] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10ab06ff221f433890e068f5ba0a9304 in term 1.
I20260929 19:14:04.999782 28836 leader_election.cc:302] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10ab06ff221f433890e068f5ba0a9304, 716c990e2a734a7d86699693fe055b4b; no voters:
I20260929 19:14:04.999902 29231 raft_consensus.cc:2781] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:05.000049 29231 raft_consensus.cc:686] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304 [term 1 LEADER]: Becoming Leader. State: Replica: 10ab06ff221f433890e068f5ba0a9304, State: Running, Role: LEADER
I20260929 19:14:05.000159 29231 consensus_queue.cc:237] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef6c234e33c41c0b428727d393b9d1c" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45249 } } peers { permanent_uuid: "716c990e2a734a7d86699693fe055b4b" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 42897 } } peers { permanent_uuid: "10ab06ff221f433890e068f5ba0a9304" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 40625 } }
I20260929 19:14:05.001093 28760 catalog_manager.cc:5495] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10ab06ff221f433890e068f5ba0a9304 (127.9.236.1). New cstate: current_term: 1 leader_uuid: "10ab06ff221f433890e068f5ba0a9304" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ef6c234e33c41c0b428727d393b9d1c" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "716c990e2a734a7d86699693fe055b4b" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 42897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10ab06ff221f433890e068f5ba0a9304" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 40625 } health_report { overall_health: HEALTHY } } }
I20260929 19:14:05.048542 29220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:05.046026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37792 (local address 127.9.236.1:40625)
0929 19:14:05.046080 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:05.046082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:05.046093 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:14:05.046163 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:05.046163 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:05.046187 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:05.046263 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:05.046412 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:05.046900 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:05.047686 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:05.047916 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:05.047983 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:05.048197 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:05.048291 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:05.048371 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:05.048465 (+ 94us) server_negotiation.cc:299] Negotiation successful
0929 19:14:05.048477 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 19:14:05.050345 29033 raft_consensus.cc:1261] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c [term 1 FOLLOWER]: Refusing update from remote peer 10ab06ff221f433890e068f5ba0a9304: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:14:05.050374 29164 raft_consensus.cc:1261] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b [term 1 FOLLOWER]: Refusing update from remote peer 10ab06ff221f433890e068f5ba0a9304: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:14:05.050717 29231 consensus_queue.cc:1035] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ef6c234e33c41c0b428727d393b9d1c" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:14:05.050882 29235 consensus_queue.cc:1035] T 06aa0e926bd7499d86f41acff7496132 P 10ab06ff221f433890e068f5ba0a9304 [LEADER]: Connected to new peer: Peer: permanent_uuid: "716c990e2a734a7d86699693fe055b4b" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:14:05.053927 29240 mvcc.cc:205] Tried to move back new op lower bound from 7334745067723603968 to 7334745067521486848. Current Snapshot: MvccSnapshot[applied={T|T < 7334745067723603968}]
I20260929 19:14:05.053927 29239 mvcc.cc:205] Tried to move back new op lower bound from 7334745067723603968 to 7334745067521486848. Current Snapshot: MvccSnapshot[applied={T|T < 7334745067723603968}]
I20260929 19:14:05.056547 28759 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:14:05.056639 28759 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:14:05.057533 28759 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f85502be0424992b83f96db5aa3967b: Sending DeleteTablet for 3 replicas of tablet 06aa0e926bd7499d86f41acff7496132
I20260929 19:14:05.057909 29144 tablet_service.cc:1514] Processing DeleteTablet for tablet 06aa0e926bd7499d86f41acff7496132 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:14:05 UTC) from {username='slave'} at 127.0.0.1:38798
I20260929 19:14:05.057886 28882 tablet_service.cc:1514] Processing DeleteTablet for tablet 06aa0e926bd7499d86f41acff7496132 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:14:05 UTC) from {username='slave'} at 127.0.0.1:37786
I20260929 19:14:05.057919 29013 tablet_service.cc:1514] Processing DeleteTablet for tablet 06aa0e926bd7499d86f41acff7496132 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:14:05 UTC) from {username='slave'} at 127.0.0.1:55040
I20260929 19:14:05.058157 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 28820
I20260929 19:14:05.059693 29249 tablet_replica.cc:330] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c: stopping tablet replica
I20260929 19:14:05.059823 29249 raft_consensus.cc:2229] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:14:05.059950 29249 raft_consensus.cc:2258] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:14:05.060137 29248 tablet_replica.cc:330] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b: stopping tablet replica
I20260929 19:14:05.060248 29248 raft_consensus.cc:2229] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:14:05.060351 29248 raft_consensus.cc:2258] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:14:05.060657 29249 ts_tablet_manager.cc:1905] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:14:05.060829 29248 ts_tablet_manager.cc:1905] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:14:05.062353 29249 ts_tablet_manager.cc:1918] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:14:05.062422 29249 log.cc:1196] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/06aa0e926bd7499d86f41acff7496132
I20260929 19:14:05.062655 29249 ts_tablet_manager.cc:1939] T 06aa0e926bd7499d86f41acff7496132 P 6ef6c234e33c41c0b428727d393b9d1c: Deleting consensus metadata
I20260929 19:14:05.062706 29248 ts_tablet_manager.cc:1918] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:14:05.062791 29248 log.cc:1196] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/06aa0e926bd7499d86f41acff7496132
I20260929 19:14:05.063042 29248 ts_tablet_manager.cc:1939] T 06aa0e926bd7499d86f41acff7496132 P 716c990e2a734a7d86699693fe055b4b: Deleting consensus metadata
I20260929 19:14:05.063165 28745 catalog_manager.cc:4841] TS 6ef6c234e33c41c0b428727d393b9d1c (127.9.236.2:45249): tablet 06aa0e926bd7499d86f41acff7496132 (table test-table [id=e46d32bcd9b64606a65c631fc5c047bd]) successfully deleted
I20260929 19:14:05.063442 28745 catalog_manager.cc:4841] TS 716c990e2a734a7d86699693fe055b4b (127.9.236.3:42897): tablet 06aa0e926bd7499d86f41acff7496132 (table test-table [id=e46d32bcd9b64606a65c631fc5c047bd]) successfully deleted
W20260929 19:14:05.064694 28747 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.236.1:40625 (error 108)
I20260929 19:14:05.064929 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 28951
I20260929 19:14:05.065528 29222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:05.065129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:40625 (local address 127.0.0.1:37804)
0929 19:14:05.065447 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:05.065488 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.1:40625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294}
W20260929 19:14:05.065665 28747 catalog_manager.cc:4584] TS 10ab06ff221f433890e068f5ba0a9304 (127.9.236.1:40625): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 06aa0e926bd7499d86f41acff7496132: Network error: Client connection negotiation failed: client connection to 127.9.236.1:40625: connect: Connection refused (error 111)
I20260929 19:14:05.070390 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 29082
I20260929 19:14:05.074772 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 28729
2026-09-29T19:14:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (926 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:05 dist-test-slave-wftq krb5kdc[29254](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:05 dist-test-slave-wftq krb5kdc[29254](info): set up 2 sockets
Sep 29 19:14:05 dist-test-slave-wftq krb5kdc[29254](info): commencing operation
krb5kdc: starting...
W20260929 19:14:06.032764 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:06 dist-test-slave-wftq krb5kdc[29254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:14:06.155445 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:46267
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:38985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:46267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:06.237641 29267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:06.237844 29267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:06.237874 29267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:06.238981 29267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:06.239023 29267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:06.239041 29267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:06.239058 29267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:06.239073 29267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:06.240517 29267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:38985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:46267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:46267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:06.240741 29267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:06.240909 29267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:06.245694 29275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:06.245744 29267 server_base.cc:1034] running on GCE node
W20260929 19:14:06.245720 29272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:06.245720 29273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:06.246270 29267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:06.246568 29267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:06.247735 29267 hybrid_clock.cc:648] HybridClock initialized: now 1790709246247710 us; error 44 us; skew 500 ppm
Sep 29 19:14:06 dist-test-slave-wftq krb5kdc[29254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709246, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:06.250157 29267 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:14:06.250586 29267 webserver.cc:460] Webserver started at http://127.9.236.62:44599/ using document root <none> and password file <none>
I20260929 19:14:06.250770 29267 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:06.250811 29267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:06.250901 29267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:06.251894 29267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "f13bfe877808486d9ea4c1dc69d2f398"
format_stamp: "Formatted at 2026-09-29 19:14:06 on dist-test-slave-wftq"
server_key: "659fc8a807e9defa08e78e71577afce2"
server_key_iv: "53ec72fd35d1c5e4f9b2427a97f1b9b1"
server_key_version: "encryptionkey@0"
I20260929 19:14:06.252187 29267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "f13bfe877808486d9ea4c1dc69d2f398"
format_stamp: "Formatted at 2026-09-29 19:14:06 on dist-test-slave-wftq"
server_key: "659fc8a807e9defa08e78e71577afce2"
server_key_iv: "53ec72fd35d1c5e4f9b2427a97f1b9b1"
server_key_version: "encryptionkey@0"
I20260929 19:14:06.253562 29267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:14:06.254276 29282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:06.254484 29267 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:14:06.254570 29267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "f13bfe877808486d9ea4c1dc69d2f398"
format_stamp: "Formatted at 2026-09-29 19:14:06 on dist-test-slave-wftq"
server_key: "659fc8a807e9defa08e78e71577afce2"
server_key_iv: "53ec72fd35d1c5e4f9b2427a97f1b9b1"
server_key_version: "encryptionkey@0"
I20260929 19:14:06.254640 29267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:06.273644 29267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:06.275447 29267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:06.275558 29267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:06.279579 29267 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:46267
I20260929 19:14:06.279666 29334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:46267 every 8 connection(s)
I20260929 19:14:06.279958 29267 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:06.280958 29335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:06.280964 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 29267
I20260929 19:14:06.281046 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:06.281260 10160 external_mini_cluster.cc:1428] Setting key 4fb5e2822dc3f4d022cda45b7d50d6c8
I20260929 19:14:06.284654 29335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398: Bootstrap starting.
I20260929 19:14:06.285398 29335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:06.285701 29335 log.cc:824] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:06.286513 29335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398: No bootstrap required, opened a new log
Sep 29 19:14:06 dist-test-slave-wftq krb5kdc[29254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:14:06.288894 29335 raft_consensus.cc:348] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f13bfe877808486d9ea4c1dc69d2f398" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46267 } }
I20260929 19:14:06.289088 29335 raft_consensus.cc:374] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:06.289165 29335 raft_consensus.cc:729] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f13bfe877808486d9ea4c1dc69d2f398, State: Initialized, Role: FOLLOWER
I20260929 19:14:06.289304 29335 consensus_queue.cc:260] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f13bfe877808486d9ea4c1dc69d2f398" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46267 } }
I20260929 19:14:06.289371 29335 raft_consensus.cc:388] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:06.289400 29335 raft_consensus.cc:482] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:06.289439 29335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:06.290141 29335 raft_consensus.cc:504] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f13bfe877808486d9ea4c1dc69d2f398" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46267 } }
I20260929 19:14:06.290266 29335 leader_election.cc:302] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f13bfe877808486d9ea4c1dc69d2f398; no voters:
I20260929 19:14:06.290439 29335 leader_election.cc:288] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:06.290515 29340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:06.290746 29335 sys_catalog.cc:564] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:06.290771 29340 raft_consensus.cc:686] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398 [term 1 LEADER]: Becoming Leader. State: Replica: f13bfe877808486d9ea4c1dc69d2f398, State: Running, Role: LEADER
I20260929 19:14:06.290944 29340 consensus_queue.cc:237] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f13bfe877808486d9ea4c1dc69d2f398" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46267 } }
I20260929 19:14:06.291338 29340 sys_catalog.cc:455] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f13bfe877808486d9ea4c1dc69d2f398. Latest consensus state: current_term: 1 leader_uuid: "f13bfe877808486d9ea4c1dc69d2f398" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f13bfe877808486d9ea4c1dc69d2f398" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46267 } } }
I20260929 19:14:06.291352 29341 sys_catalog.cc:455] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f13bfe877808486d9ea4c1dc69d2f398" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f13bfe877808486d9ea4c1dc69d2f398" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 46267 } } }
I20260929 19:14:06.291514 29340 sys_catalog.cc:458] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:06.291520 29341 sys_catalog.cc:458] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:06.291851 29349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:06.292457 29349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:06.293069 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:06.281820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38106 (local address 127.9.236.62:46267)
0929 19:14:06.282075 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:06.282086 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:06.282123 (+ 37us) server_negotiation.cc:407] Connection header received
0929 19:14:06.282674 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:06.282685 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:06.282754 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:06.282925 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:06.283484 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.284874 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.285657 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.286177 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.286289 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:06.288903 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:06.288908 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:06.288908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:06.288920 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:06.290797 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:06.291245 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:06.291249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:06.291250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:06.291361 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:06.291591 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:06.291591 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:06.291592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:06.291915 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:06.292558 (+ 643us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:06.292631 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 19:14:06.292856 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":106,"threads_started":1}
I20260929 19:14:06.294394 29349 catalog_manager.cc:1347] Generated new cluster ID: ad8c77cfeb8a4c1e9e5e9906950672be
I20260929 19:14:06.294452 29349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:06.325126 29349 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:06.325886 29349 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:06.330929 29349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398: Generated new TSK 0
I20260929 19:14:06.331108 29349 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.236.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
Entry for principal kudu/127.9.236.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab.
I20260929 19:14:06.360047 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:46267
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:38985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:06.446775 29361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:06.446992 29361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:06.447022 29361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:06.448181 29361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:06.448221 29361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:06.448256 29361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:14:06.449747 29361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:38985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.1.keytab
--principal=kudu/127.9.236.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:46267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:06.449978 29361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:06.450192 29361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:06.455312 29366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:06.455312 29367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:06.455391 29369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:06.455755 29361 server_base.cc:1034] running on GCE node
I20260929 19:14:06.455998 29361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:06.456301 29361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:06.457446 29361 hybrid_clock.cc:648] HybridClock initialized: now 1790709246457424 us; error 43 us; skew 500 ppm
Sep 29 19:14:06 dist-test-slave-wftq krb5kdc[29254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709246, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:06.460081 29361 init.cc:377] Logged in from keytab as kudu/127.9.236.1@KRBTEST.COM (short username kudu)
I20260929 19:14:06.460537 29361 webserver.cc:460] Webserver started at http://127.9.236.1:46737/ using document root <none> and password file <none>
I20260929 19:14:06.460727 29361 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:06.460769 29361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:06.460866 29361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:06.461957 29361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "dcf8e641399a4cdda74db6a3369333ac"
format_stamp: "Formatted at 2026-09-29 19:14:06 on dist-test-slave-wftq"
server_key: "35db2d846a492a52503bc22d410adf11"
server_key_iv: "945671e0b4e638efef8e353bd8147bb9"
server_key_version: "encryptionkey@0"
I20260929 19:14:06.462261 29361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "dcf8e641399a4cdda74db6a3369333ac"
format_stamp: "Formatted at 2026-09-29 19:14:06 on dist-test-slave-wftq"
server_key: "35db2d846a492a52503bc22d410adf11"
server_key_iv: "945671e0b4e638efef8e353bd8147bb9"
server_key_version: "encryptionkey@0"
I20260929 19:14:06.463511 29361 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 19:14:06.464193 29376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:06.464378 29361 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260929 19:14:06.464468 29361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "dcf8e641399a4cdda74db6a3369333ac"
format_stamp: "Formatted at 2026-09-29 19:14:06 on dist-test-slave-wftq"
server_key: "35db2d846a492a52503bc22d410adf11"
server_key_iv: "945671e0b4e638efef8e353bd8147bb9"
server_key_version: "encryptionkey@0"
I20260929 19:14:06.464529 29361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:06.481859 29361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:06.483735 29361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:06.483877 29361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:06.484083 29361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:14:06.484376 29361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:14:06.484409 29361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:06.484436 29361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:14:06.484453 29361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:06.490486 29361 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:32835
I20260929 19:14:06.490551 29489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:32835 every 8 connection(s)
I20260929 19:14:06.490968 29361 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:14:06.496423 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 29361
I20260929 19:14:06.496556 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:14:06.496781 10160 external_mini_cluster.cc:1428] Setting key 1ff107ae406300787a11e8076b20f53b
Sep 29 19:14:06 dist-test-slave-wftq krb5kdc[29254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709246, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.1@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:14:06.501299 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:06.491826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:55897 (local address 127.9.236.62:46267)
0929 19:14:06.491897 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:06.491899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:06.492159 (+ 260us) server_negotiation.cc:407] Connection header received
0929 19:14:06.492713 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:06.492714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:06.492724 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:06.492791 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:06.493912 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.494431 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.495472 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.495680 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.495740 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:06.498150 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:06.498151 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:06.498152 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:06.498158 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:06.499542 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:06.500311 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:06.500312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:06.500312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:06.500363 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:06.500638 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:06.500638 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:06.500639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:06.500751 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:06.500802 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:06.501141 (+ 339us) server_negotiation.cc:299] Negotiation successful
0929 19:14:06.501226 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 19:14:06.501536 29492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:06.491779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46267 (local address 127.9.236.1:55897)
0929 19:14:06.491982 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:06.492049 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:06.492608 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:06.492860 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:06.492863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:06.493576 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:06.493809 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:06.493814 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.494511 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.494512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:06.495393 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:06.495395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.495751 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.495752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:06.495854 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:06.496379 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:06.496391 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:06.498031 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:06.499665 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:06.499675 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:06.499678 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:06.500199 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:06.500425 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:06.500426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:06.500426 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:06.500567 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:06.500812 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:06.500817 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:06.500984 (+ 167us) client_negotiation.cc:769] Sending connection context
0929 19:14:06.501125 (+ 141us) client_negotiation.cc:240] Negotiation successful
0929 19:14:06.501319 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":124,"threads_started":1}
I20260929 19:14:06.502122 29490 heartbeater.cc:347] Connected to a master server at 127.9.236.62:46267
I20260929 19:14:06.502226 29490 heartbeater.cc:464] Registering TS with master...
I20260929 19:14:06.502411 29490 heartbeater.cc:511] Master 127.9.236.62:46267 requested a full tablet report, sending...
I20260929 19:14:06.503175 29299 ts_manager.cc:194] Registered new tserver with Master: dcf8e641399a4cdda74db6a3369333ac (127.9.236.1:32835)
I20260929 19:14:06.504096 29299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.1@KRBTEST.COM'} at 127.9.236.1:55897
WARNING: no policy specified for kudu/127.9.236.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
Entry for principal kudu/127.9.236.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab.
I20260929 19:14:06.524061 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:46267
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:38985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:06.607548 29495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:06.607760 29495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:06.607782 29495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:06.608870 29495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:06.608911 29495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:06.608935 29495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:14:06.610320 29495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:38985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.2.keytab
--principal=kudu/127.9.236.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:46267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:06.610496 29495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:06.610677 29495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:06.615595 29500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:06.615674 29495 server_base.cc:1034] running on GCE node
W20260929 19:14:06.615638 29503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:06.615638 29501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:06.616055 29495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:06.616314 29495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:06.617455 29495 hybrid_clock.cc:648] HybridClock initialized: now 1790709246617445 us; error 32 us; skew 500 ppm
Sep 29 19:14:06 dist-test-slave-wftq krb5kdc[29254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709246, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:06.619987 29495 init.cc:377] Logged in from keytab as kudu/127.9.236.2@KRBTEST.COM (short username kudu)
I20260929 19:14:06.620443 29495 webserver.cc:460] Webserver started at http://127.9.236.2:36915/ using document root <none> and password file <none>
I20260929 19:14:06.620666 29495 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:06.620715 29495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:06.620841 29495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:06.622007 29495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "a0dba199198842c5a3f00a4a13d57c12"
format_stamp: "Formatted at 2026-09-29 19:14:06 on dist-test-slave-wftq"
server_key: "c2ee08f2fa3641a29c13a5caef428eec"
server_key_iv: "d8570a3472aa4611f020a227088fd56d"
server_key_version: "encryptionkey@0"
I20260929 19:14:06.622351 29495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "a0dba199198842c5a3f00a4a13d57c12"
format_stamp: "Formatted at 2026-09-29 19:14:06 on dist-test-slave-wftq"
server_key: "c2ee08f2fa3641a29c13a5caef428eec"
server_key_iv: "d8570a3472aa4611f020a227088fd56d"
server_key_version: "encryptionkey@0"
I20260929 19:14:06.623836 29495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 19:14:06.624728 29510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:06.624977 29495 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:14:06.625067 29495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "a0dba199198842c5a3f00a4a13d57c12"
format_stamp: "Formatted at 2026-09-29 19:14:06 on dist-test-slave-wftq"
server_key: "c2ee08f2fa3641a29c13a5caef428eec"
server_key_iv: "d8570a3472aa4611f020a227088fd56d"
server_key_version: "encryptionkey@0"
I20260929 19:14:06.625165 29495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:06.641717 29495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:06.643906 29495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:06.644024 29495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:06.644232 29495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:14:06.644517 29495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:14:06.644549 29495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:06.644572 29495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:14:06.644583 29495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:06.651363 29495 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:40545
I20260929 19:14:06.651453 29623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:40545 every 8 connection(s)
I20260929 19:14:06.651767 29495 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
Sep 29 19:14:06 dist-test-slave-wftq krb5kdc[29254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709246, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.2@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:14:06.659509 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 29495
I20260929 19:14:06.659631 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:14:06.659852 10160 external_mini_cluster.cc:1428] Setting key e8c422d8d01c6b88b6398fe0c568a4c6
I20260929 19:14:06.661594 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:06.652522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:56533 (local address 127.9.236.62:46267)
0929 19:14:06.652583 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:06.652585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:06.652782 (+ 197us) server_negotiation.cc:407] Connection header received
0929 19:14:06.653353 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:06.653354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:06.653365 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:06.653443 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:06.654383 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.654924 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.656000 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.656191 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.656245 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:06.658394 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:06.658396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:06.658396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:06.658402 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:06.659999 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:06.660522 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:06.660523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:06.660524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:06.660578 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:06.660894 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:06.660896 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:06.660896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:06.661045 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:06.661156 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:06.661398 (+ 242us) server_negotiation.cc:299] Negotiation successful
0929 19:14:06.661494 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 19:14:06.661716 29626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:06.652459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46267 (local address 127.9.236.2:56533)
0929 19:14:06.652650 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:06.652692 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:06.653259 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:06.653493 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:06.653496 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:06.654084 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:06.654290 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:06.654294 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.655035 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.655036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:06.655903 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:06.655906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.656258 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.656259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:06.656354 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:06.656934 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:06.656942 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:06.658314 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:06.660155 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:06.660163 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:06.660166 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:06.660438 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:06.660685 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:06.660685 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:06.660686 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:06.660823 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:06.661136 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:06.661148 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:06.661270 (+ 122us) client_negotiation.cc:769] Sending connection context
0929 19:14:06.661358 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 19:14:06.661484 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":89,"threads_started":1}
I20260929 19:14:06.662431 29624 heartbeater.cc:347] Connected to a master server at 127.9.236.62:46267
I20260929 19:14:06.662559 29624 heartbeater.cc:464] Registering TS with master...
I20260929 19:14:06.662778 29624 heartbeater.cc:511] Master 127.9.236.62:46267 requested a full tablet report, sending...
I20260929 19:14:06.663199 29299 ts_manager.cc:194] Registered new tserver with Master: a0dba199198842c5a3f00a4a13d57c12 (127.9.236.2:40545)
I20260929 19:14:06.663596 29299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.2@KRBTEST.COM'} at 127.9.236.2:56533
WARNING: no policy specified for kudu/127.9.236.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
Entry for principal kudu/127.9.236.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab.
I20260929 19:14:06.685642 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:46267
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:38985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:06.767802 29629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:06.768038 29629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:06.768061 29629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:06.769384 29629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:06.769429 29629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:06.769455 29629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:14:06.770941 29629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:38985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.3.keytab
--principal=kudu/127.9.236.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:46267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:06.771178 29629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:06.771401 29629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:06.776582 29635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:06.776541 29634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:06.776542 29637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:06.776831 29629 server_base.cc:1034] running on GCE node
I20260929 19:14:06.777048 29629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:06.777325 29629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:06.778474 29629 hybrid_clock.cc:648] HybridClock initialized: now 1790709246778454 us; error 38 us; skew 500 ppm
Sep 29 19:14:06 dist-test-slave-wftq krb5kdc[29254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709246, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:06.781260 29629 init.cc:377] Logged in from keytab as kudu/127.9.236.3@KRBTEST.COM (short username kudu)
I20260929 19:14:06.781746 29629 webserver.cc:460] Webserver started at http://127.9.236.3:37549/ using document root <none> and password file <none>
I20260929 19:14:06.781975 29629 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:06.782023 29629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:06.782151 29629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:06.783273 29629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "49d8879b6e394d1cae1f345f77bd5b83"
format_stamp: "Formatted at 2026-09-29 19:14:06 on dist-test-slave-wftq"
server_key: "260a2a86e1eacd4bd9e5749c25301465"
server_key_iv: "8e0c4928a7b8cbbd7ed5eb72df52f39e"
server_key_version: "encryptionkey@0"
I20260929 19:14:06.783646 29629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "49d8879b6e394d1cae1f345f77bd5b83"
format_stamp: "Formatted at 2026-09-29 19:14:06 on dist-test-slave-wftq"
server_key: "260a2a86e1eacd4bd9e5749c25301465"
server_key_iv: "8e0c4928a7b8cbbd7ed5eb72df52f39e"
server_key_version: "encryptionkey@0"
I20260929 19:14:06.784992 29629 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:14:06.785820 29644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:06.786005 29629 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:06.786090 29629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "49d8879b6e394d1cae1f345f77bd5b83"
format_stamp: "Formatted at 2026-09-29 19:14:06 on dist-test-slave-wftq"
server_key: "260a2a86e1eacd4bd9e5749c25301465"
server_key_iv: "8e0c4928a7b8cbbd7ed5eb72df52f39e"
server_key_version: "encryptionkey@0"
I20260929 19:14:06.786161 29629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:06.809386 29629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:06.811657 29629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:06.811793 29629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:06.812055 29629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:14:06.812379 29629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:14:06.812424 29629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:06.812448 29629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:14:06.812459 29629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:06.819330 29629 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:40923
I20260929 19:14:06.819392 29757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:40923 every 8 connection(s)
I20260929 19:14:06.819726 29629 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:14:06.821434 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 29629
I20260929 19:14:06.821532 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:14:06.821712 10160 external_mini_cluster.cc:1428] Setting key 0c2000accbc0e761f3cf5eb60f1a3e4f
Sep 29 19:14:06 dist-test-slave-wftq krb5kdc[29254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709246, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.3@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:14:06.829558 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:06.820524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:54005 (local address 127.9.236.62:46267)
0929 19:14:06.820605 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:06.820608 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:06.820868 (+ 260us) server_negotiation.cc:407] Connection header received
0929 19:14:06.821464 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:06.821466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:06.821480 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:06.821650 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:06.822778 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.823323 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.824355 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.824563 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.824618 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:06.826836 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:06.826837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:06.826838 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:06.826845 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:06.828114 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:06.828553 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:06.828554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:06.828555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:06.828613 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:06.828838 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:06.828839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:06.828839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:06.828975 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:06.829035 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:06.829364 (+ 329us) server_negotiation.cc:299] Negotiation successful
0929 19:14:06.829453 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 19:14:06.829779 29760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:06.820476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:46267 (local address 127.9.236.3:54005)
0929 19:14:06.820684 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:06.820733 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:06.821344 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:06.821738 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:06.821742 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:06.822436 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:06.822692 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:06.822698 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.823416 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.823417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:06.824268 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:06.824270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.824645 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.824646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:06.824770 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:06.825296 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:06.825305 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:06.826746 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:06.828192 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:06.828198 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:06.828201 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:06.828484 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:06.828667 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:06.828668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:06.828668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:06.828775 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:06.829033 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:06.829040 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:06.829238 (+ 198us) client_negotiation.cc:769] Sending connection context
0929 19:14:06.829336 (+ 98us) client_negotiation.cc:240] Negotiation successful
0929 19:14:06.829526 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":111,"threads_started":1}
I20260929 19:14:06.830319 29758 heartbeater.cc:347] Connected to a master server at 127.9.236.62:46267
I20260929 19:14:06.830415 29758 heartbeater.cc:464] Registering TS with master...
I20260929 19:14:06.830595 29758 heartbeater.cc:511] Master 127.9.236.62:46267 requested a full tablet report, sending...
I20260929 19:14:06.831133 29299 ts_manager.cc:194] Registered new tserver with Master: 49d8879b6e394d1cae1f345f77bd5b83 (127.9.236.3:40923)
I20260929 19:14:06.831643 29299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.236.3@KRBTEST.COM'} at 127.9.236.3:54005
I20260929 19:14:06.833418 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:14:06.840500 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:06.834916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38116 (local address 127.9.236.62:46267)
0929 19:14:06.834993 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:06.834995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:06.835086 (+ 91us) server_negotiation.cc:407] Connection header received
0929 19:14:06.835158 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:06.835159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:06.835169 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:06.835227 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:06.835733 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.836262 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.836949 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.837194 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.837253 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:06.838396 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:06.838397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:06.838398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:06.838404 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:06.839519 (+ 1115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:06.839810 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:06.839811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:06.839812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:06.839863 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:06.840018 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:06.840019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:06.840019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:06.840160 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:06.840255 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:06.840343 (+ 88us) server_negotiation.cc:299] Negotiation successful
0929 19:14:06.840412 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 19:14:06.842094 29299 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:14:06.843822 29299 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:14:06.850373 29773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:06.846521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56878 (local address 127.9.236.1:32835)
0929 19:14:06.846753 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:06.846759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:06.846820 (+ 61us) server_negotiation.cc:407] Connection header received
0929 19:14:06.846927 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:06.846930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:06.846950 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:06.847041 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:06.847243 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.848049 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.849258 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.849978 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.850056 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:06.850116 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:06.850277 (+ 161us) server_negotiation.cc:299] Negotiation successful
0929 19:14:06.850307 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":111,"threads_started":1}
I20260929 19:14:06.850425 29770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:06.846341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:32835 (local address 127.0.0.1:56878)
0929 19:14:06.846731 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:06.846742 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:06.846810 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:06.847062 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:06.847065 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:06.847069 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:06.847135 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:06.847137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.848144 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.848145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:06.849158 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:06.849161 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.850064 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.850065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:06.850158 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:06.850175 (+ 17us) client_negotiation.cc:769] Sending connection context
0929 19:14:06.850262 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 19:14:06.850326 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":92,"threads_started":1}
I20260929 19:14:06.850800 29769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:06.846320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:40545 (local address 127.0.0.1:41484)
0929 19:14:06.846498 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:06.846535 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:06.846597 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:06.847200 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:06.847201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:06.847204 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:06.847275 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:06.847278 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.848742 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.848744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:06.849690 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:06.849692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.850568 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.850569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:06.850634 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:06.850637 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:14:06.850683 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 19:14:06.850731 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":2,"thread_start_us":74,"threads_started":1}
I20260929 19:14:06.851111 29771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:06.846426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41484 (local address 127.9.236.2:40545)
0929 19:14:06.846953 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:06.846958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:06.846967 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:14:06.847012 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:06.847015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:06.847030 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:06.847102 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:06.847393 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.848594 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.849768 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.850402 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.850466 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:06.850528 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:06.850974 (+ 446us) server_negotiation.cc:299] Negotiation successful
0929 19:14:06.851026 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":84,"threads_started":1}
I20260929 19:14:06.851145 29424 tablet_service.cc:1467] Processing CreateTablet for tablet cd06ebb394854054bbc8fcff868c0b97 (DEFAULT_TABLE table=test-table [id=1d51b12194ff46ffbe9ae1604c79df4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:14:06.851536 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd06ebb394854054bbc8fcff868c0b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:06.851820 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:06.846539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:40923 (local address 127.0.0.1:48246)
0929 19:14:06.846820 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:06.846831 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:06.846883 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:06.847344 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:06.847345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:06.847348 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:06.847410 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:06.847412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.848715 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.848716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:06.849982 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:06.849984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.851448 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.851449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:06.851528 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:06.851534 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 19:14:06.851677 (+ 143us) client_negotiation.cc:240] Negotiation successful
0929 19:14:06.851746 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":33,"thread_start_us":93,"threads_started":1}
I20260929 19:14:06.851861 29558 tablet_service.cc:1467] Processing CreateTablet for tablet cd06ebb394854054bbc8fcff868c0b97 (DEFAULT_TABLE table=test-table [id=1d51b12194ff46ffbe9ae1604c79df4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:14:06.852137 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:06.846804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48246 (local address 127.9.236.3:40923)
0929 19:14:06.847188 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:06.847192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:06.847202 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:14:06.847237 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:06.847238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:06.847256 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:06.847341 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:06.847478 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.848593 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.850636 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:06.851341 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:06.851423 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:06.851478 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:06.851996 (+ 518us) server_negotiation.cc:299] Negotiation successful
0929 19:14:06.852054 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":104,"threads_started":1}
I20260929 19:14:06.852250 29558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd06ebb394854054bbc8fcff868c0b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:06.852875 29692 tablet_service.cc:1467] Processing CreateTablet for tablet cd06ebb394854054bbc8fcff868c0b97 (DEFAULT_TABLE table=test-table [id=1d51b12194ff46ffbe9ae1604c79df4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:14:06.853330 29692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd06ebb394854054bbc8fcff868c0b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:06.854956 29775 tablet_bootstrap.cc:492] T cd06ebb394854054bbc8fcff868c0b97 P dcf8e641399a4cdda74db6a3369333ac: Bootstrap starting.
I20260929 19:14:06.855679 29775 tablet_bootstrap.cc:654] T cd06ebb394854054bbc8fcff868c0b97 P dcf8e641399a4cdda74db6a3369333ac: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:06.856158 29775 log.cc:824] T cd06ebb394854054bbc8fcff868c0b97 P dcf8e641399a4cdda74db6a3369333ac: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:06.856268 29777 tablet_bootstrap.cc:492] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83: Bootstrap starting.
I20260929 19:14:06.856552 29776 tablet_bootstrap.cc:492] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12: Bootstrap starting.
I20260929 19:14:06.856894 29775 tablet_bootstrap.cc:492] T cd06ebb394854054bbc8fcff868c0b97 P dcf8e641399a4cdda74db6a3369333ac: No bootstrap required, opened a new log
I20260929 19:14:06.856959 29775 ts_tablet_manager.cc:1397] T cd06ebb394854054bbc8fcff868c0b97 P dcf8e641399a4cdda74db6a3369333ac: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:14:06.856992 29777 tablet_bootstrap.cc:654] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:06.857250 29776 tablet_bootstrap.cc:654] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:06.857621 29777 log.cc:824] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:06.857846 29776 log.cc:824] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:06.858292 29777 tablet_bootstrap.cc:492] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83: No bootstrap required, opened a new log
I20260929 19:14:06.858348 29777 ts_tablet_manager.cc:1397] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 19:14:06.858856 29775 raft_consensus.cc:348] T cd06ebb394854054bbc8fcff868c0b97 P dcf8e641399a4cdda74db6a3369333ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0dba199198842c5a3f00a4a13d57c12" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 40545 } } peers { permanent_uuid: "dcf8e641399a4cdda74db6a3369333ac" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 32835 } } peers { permanent_uuid: "49d8879b6e394d1cae1f345f77bd5b83" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40923 } }
I20260929 19:14:06.859017 29775 raft_consensus.cc:374] T cd06ebb394854054bbc8fcff868c0b97 P dcf8e641399a4cdda74db6a3369333ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:06.859050 29775 raft_consensus.cc:729] T cd06ebb394854054bbc8fcff868c0b97 P dcf8e641399a4cdda74db6a3369333ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcf8e641399a4cdda74db6a3369333ac, State: Initialized, Role: FOLLOWER
I20260929 19:14:06.859172 29775 consensus_queue.cc:260] T cd06ebb394854054bbc8fcff868c0b97 P dcf8e641399a4cdda74db6a3369333ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0dba199198842c5a3f00a4a13d57c12" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 40545 } } peers { permanent_uuid: "dcf8e641399a4cdda74db6a3369333ac" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 32835 } } peers { permanent_uuid: "49d8879b6e394d1cae1f345f77bd5b83" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40923 } }
I20260929 19:14:06.859457 29775 ts_tablet_manager.cc:1428] T cd06ebb394854054bbc8fcff868c0b97 P dcf8e641399a4cdda74db6a3369333ac: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 19:14:06.859503 29490 heartbeater.cc:503] Master 127.9.236.62:46267 was elected leader, sending a full tablet report...
I20260929 19:14:06.859851 29777 raft_consensus.cc:348] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0dba199198842c5a3f00a4a13d57c12" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 40545 } } peers { permanent_uuid: "dcf8e641399a4cdda74db6a3369333ac" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 32835 } } peers { permanent_uuid: "49d8879b6e394d1cae1f345f77bd5b83" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40923 } }
I20260929 19:14:06.860014 29777 raft_consensus.cc:374] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:06.860059 29777 raft_consensus.cc:729] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49d8879b6e394d1cae1f345f77bd5b83, State: Initialized, Role: FOLLOWER
I20260929 19:14:06.860172 29777 consensus_queue.cc:260] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0dba199198842c5a3f00a4a13d57c12" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 40545 } } peers { permanent_uuid: "dcf8e641399a4cdda74db6a3369333ac" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 32835 } } peers { permanent_uuid: "49d8879b6e394d1cae1f345f77bd5b83" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40923 } }
I20260929 19:14:06.860440 29777 ts_tablet_manager.cc:1428] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:14:06.860591 29758 heartbeater.cc:503] Master 127.9.236.62:46267 was elected leader, sending a full tablet report...
I20260929 19:14:06.861048 29776 tablet_bootstrap.cc:492] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12: No bootstrap required, opened a new log
I20260929 19:14:06.861133 29776 ts_tablet_manager.cc:1397] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260929 19:14:06.862658 29776 raft_consensus.cc:348] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0dba199198842c5a3f00a4a13d57c12" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 40545 } } peers { permanent_uuid: "dcf8e641399a4cdda74db6a3369333ac" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 32835 } } peers { permanent_uuid: "49d8879b6e394d1cae1f345f77bd5b83" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40923 } }
I20260929 19:14:06.862788 29776 raft_consensus.cc:374] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:06.862819 29776 raft_consensus.cc:729] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0dba199198842c5a3f00a4a13d57c12, State: Initialized, Role: FOLLOWER
I20260929 19:14:06.862926 29776 consensus_queue.cc:260] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0dba199198842c5a3f00a4a13d57c12" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 40545 } } peers { permanent_uuid: "dcf8e641399a4cdda74db6a3369333ac" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 32835 } } peers { permanent_uuid: "49d8879b6e394d1cae1f345f77bd5b83" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40923 } }
I20260929 19:14:06.863180 29776 ts_tablet_manager.cc:1428] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:14:06.863273 29624 heartbeater.cc:503] Master 127.9.236.62:46267 was elected leader, sending a full tablet report...
W20260929 19:14:06.902372 29625 tablet.cc:2369] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 19:14:06.991791 29491 tablet.cc:2369] T cd06ebb394854054bbc8fcff868c0b97 P dcf8e641399a4cdda74db6a3369333ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:14:07.070140 29782 raft_consensus.cc:482] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:14:07.070299 29782 raft_consensus.cc:504] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0dba199198842c5a3f00a4a13d57c12" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 40545 } } peers { permanent_uuid: "dcf8e641399a4cdda74db6a3369333ac" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 32835 } } peers { permanent_uuid: "49d8879b6e394d1cae1f345f77bd5b83" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40923 } }
W20260929 19:14:07.070400 29759 tablet.cc:2369] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 19:14:07.070770 29782 leader_election.cc:288] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0dba199198842c5a3f00a4a13d57c12 (127.9.236.2:40545), dcf8e641399a4cdda74db6a3369333ac (127.9.236.1:32835)
I20260929 19:14:07.074832 29760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.070944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:40545 (local address 127.9.236.3:58385)
0929 19:14:07.071015 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:07.071026 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:07.071079 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:07.071315 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:07.071316 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:07.071318 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:07.071414 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:07.071416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.072403 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.072404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:07.073694 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:07.073697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.074514 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.074515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:07.074607 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.074609 (+ 2us) client_negotiation.cc:769] Sending connection context
0929 19:14:07.074679 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 19:14:07.074721 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":9}
I20260929 19:14:07.074842 29771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.071062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:58385 (local address 127.9.236.2:40545)
0929 19:14:07.071141 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:07.071143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:07.071157 (+ 14us) server_negotiation.cc:407] Connection header received
0929 19:14:07.071197 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:07.071198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:07.071213 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:07.071298 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:07.071490 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.072302 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.073797 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.074412 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.074484 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.074500 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:07.074708 (+ 208us) server_negotiation.cc:299] Negotiation successful
0929 19:14:07.074761 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 19:14:07.074838 29784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.071052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:32835 (local address 127.9.236.3:42159)
0929 19:14:07.071296 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:07.071306 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:07.071375 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:07.071518 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:07.071519 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:07.071520 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:07.071574 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:07.071575 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.072473 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.072474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:07.073694 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:07.073697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.074514 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.074515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:07.074603 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.074609 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 19:14:07.074690 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 19:14:07.074721 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":6,"thread_start_us":158,"threads_started":1}
I20260929 19:14:07.074842 29773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.071152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:42159 (local address 127.9.236.1:32835)
0929 19:14:07.071197 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:07.071199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:07.071389 (+ 190us) server_negotiation.cc:407] Connection header received
0929 19:14:07.071432 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:07.071433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:07.071446 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:07.071503 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:07.071638 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.072374 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.073808 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.074395 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.074496 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.074507 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:07.074706 (+ 199us) server_negotiation.cc:299] Negotiation successful
0929 19:14:07.074755 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:14:07.075251 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd06ebb394854054bbc8fcff868c0b97" candidate_uuid: "49d8879b6e394d1cae1f345f77bd5b83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0dba199198842c5a3f00a4a13d57c12" is_pre_election: true
I20260929 19:14:07.075295 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd06ebb394854054bbc8fcff868c0b97" candidate_uuid: "49d8879b6e394d1cae1f345f77bd5b83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcf8e641399a4cdda74db6a3369333ac" is_pre_election: true
I20260929 19:14:07.075479 29578 raft_consensus.cc:2454] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49d8879b6e394d1cae1f345f77bd5b83 in term 0.
I20260929 19:14:07.075479 29444 raft_consensus.cc:2454] T cd06ebb394854054bbc8fcff868c0b97 P dcf8e641399a4cdda74db6a3369333ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49d8879b6e394d1cae1f345f77bd5b83 in term 0.
I20260929 19:14:07.075742 29646 leader_election.cc:302] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49d8879b6e394d1cae1f345f77bd5b83, dcf8e641399a4cdda74db6a3369333ac; no voters:
I20260929 19:14:07.075892 29782 raft_consensus.cc:2781] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:14:07.075965 29782 raft_consensus.cc:482] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:14:07.075980 29782 raft_consensus.cc:3037] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:07.076732 29782 raft_consensus.cc:504] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0dba199198842c5a3f00a4a13d57c12" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 40545 } } peers { permanent_uuid: "dcf8e641399a4cdda74db6a3369333ac" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 32835 } } peers { permanent_uuid: "49d8879b6e394d1cae1f345f77bd5b83" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40923 } }
I20260929 19:14:07.076870 29782 leader_election.cc:288] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 [CANDIDATE]: Term 1 election: Requested vote from peers a0dba199198842c5a3f00a4a13d57c12 (127.9.236.2:40545), dcf8e641399a4cdda74db6a3369333ac (127.9.236.1:32835)
I20260929 19:14:07.077085 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd06ebb394854054bbc8fcff868c0b97" candidate_uuid: "49d8879b6e394d1cae1f345f77bd5b83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0dba199198842c5a3f00a4a13d57c12"
I20260929 19:14:07.077091 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd06ebb394854054bbc8fcff868c0b97" candidate_uuid: "49d8879b6e394d1cae1f345f77bd5b83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcf8e641399a4cdda74db6a3369333ac"
I20260929 19:14:07.077195 29578 raft_consensus.cc:3037] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:07.077224 29444 raft_consensus.cc:3037] T cd06ebb394854054bbc8fcff868c0b97 P dcf8e641399a4cdda74db6a3369333ac [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:07.077955 29444 raft_consensus.cc:2454] T cd06ebb394854054bbc8fcff868c0b97 P dcf8e641399a4cdda74db6a3369333ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49d8879b6e394d1cae1f345f77bd5b83 in term 1.
I20260929 19:14:07.078095 29578 raft_consensus.cc:2454] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49d8879b6e394d1cae1f345f77bd5b83 in term 1.
I20260929 19:14:07.078117 29646 leader_election.cc:302] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49d8879b6e394d1cae1f345f77bd5b83, dcf8e641399a4cdda74db6a3369333ac; no voters:
I20260929 19:14:07.078219 29782 raft_consensus.cc:2781] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:07.078408 29782 raft_consensus.cc:686] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 [term 1 LEADER]: Becoming Leader. State: Replica: 49d8879b6e394d1cae1f345f77bd5b83, State: Running, Role: LEADER
I20260929 19:14:07.078559 29782 consensus_queue.cc:237] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0dba199198842c5a3f00a4a13d57c12" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 40545 } } peers { permanent_uuid: "dcf8e641399a4cdda74db6a3369333ac" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 32835 } } peers { permanent_uuid: "49d8879b6e394d1cae1f345f77bd5b83" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40923 } }
I20260929 19:14:07.079517 29299 catalog_manager.cc:5495] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49d8879b6e394d1cae1f345f77bd5b83 (127.9.236.3). New cstate: current_term: 1 leader_uuid: "49d8879b6e394d1cae1f345f77bd5b83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0dba199198842c5a3f00a4a13d57c12" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 40545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcf8e641399a4cdda74db6a3369333ac" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 32835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49d8879b6e394d1cae1f345f77bd5b83" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 40923 } health_report { overall_health: HEALTHY } } }
I20260929 19:14:07.134454 29774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.131755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48258 (local address 127.9.236.3:40923)
0929 19:14:07.131857 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:07.131860 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:07.131872 (+ 12us) server_negotiation.cc:407] Connection header received
0929 19:14:07.131935 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:07.131937 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:07.131974 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:07.132079 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:07.132246 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.132804 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.133681 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.133893 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.133958 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.134079 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:07.134227 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:07.134298 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:07.134318 (+ 20us) server_negotiation.cc:299] Negotiation successful
0929 19:14:07.134341 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 19:14:07.136468 29444 raft_consensus.cc:1261] T cd06ebb394854054bbc8fcff868c0b97 P dcf8e641399a4cdda74db6a3369333ac [term 1 FOLLOWER]: Refusing update from remote peer 49d8879b6e394d1cae1f345f77bd5b83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:14:07.136529 29578 raft_consensus.cc:1261] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12 [term 1 FOLLOWER]: Refusing update from remote peer 49d8879b6e394d1cae1f345f77bd5b83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:14:07.136971 29785 consensus_queue.cc:1035] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcf8e641399a4cdda74db6a3369333ac" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 32835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:14:07.137137 29782 consensus_queue.cc:1035] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0dba199198842c5a3f00a4a13d57c12" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:14:07.139564 29788 mvcc.cc:205] Tried to move back new op lower bound from 7334745076268126208 to 7334745076034445312. Current Snapshot: MvccSnapshot[applied={T|T < 7334745076268126208}]
I20260929 19:14:07.142807 29298 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:14:07.142895 29298 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:14:07.143697 29298 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f13bfe877808486d9ea4c1dc69d2f398: Sending DeleteTablet for 3 replicas of tablet cd06ebb394854054bbc8fcff868c0b97
I20260929 19:14:07.143983 29558 tablet_service.cc:1514] Processing DeleteTablet for tablet cd06ebb394854054bbc8fcff868c0b97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:14:07 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:41484
I20260929 19:14:07.144032 29692 tablet_service.cc:1514] Processing DeleteTablet for tablet cd06ebb394854054bbc8fcff868c0b97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:14:07 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:48246
I20260929 19:14:07.144022 29424 tablet_service.cc:1514] Processing DeleteTablet for tablet cd06ebb394854054bbc8fcff868c0b97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:14:07 UTC) from {username='kudu', principal='kudu/127.9.236.62@KRBTEST.COM'} at 127.0.0.1:56878
I20260929 19:14:07.144320 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 29361
I20260929 19:14:07.144364 29800 tablet_replica.cc:330] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83: stopping tablet replica
I20260929 19:14:07.144464 29800 raft_consensus.cc:2229] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:14:07.144549 29800 raft_consensus.cc:2258] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:14:07.144675 29799 tablet_replica.cc:330] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12: stopping tablet replica
I20260929 19:14:07.144742 29799 raft_consensus.cc:2229] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:14:07.144824 29799 raft_consensus.cc:2258] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:14:07.145748 29799 ts_tablet_manager.cc:1905] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:14:07.146939 29800 ts_tablet_manager.cc:1905] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:14:07.147733 29799 ts_tablet_manager.cc:1918] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:14:07.147815 29799 log.cc:1196] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/cd06ebb394854054bbc8fcff868c0b97
I20260929 19:14:07.148069 29799 ts_tablet_manager.cc:1939] T cd06ebb394854054bbc8fcff868c0b97 P a0dba199198842c5a3f00a4a13d57c12: Deleting consensus metadata
I20260929 19:14:07.148592 29800 ts_tablet_manager.cc:1918] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:14:07.148728 29800 log.cc:1196] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/cd06ebb394854054bbc8fcff868c0b97
I20260929 19:14:07.148905 29286 catalog_manager.cc:4841] TS a0dba199198842c5a3f00a4a13d57c12 (127.9.236.2:40545): tablet cd06ebb394854054bbc8fcff868c0b97 (table test-table [id=1d51b12194ff46ffbe9ae1604c79df4c]) successfully deleted
I20260929 19:14:07.149051 29800 ts_tablet_manager.cc:1939] T cd06ebb394854054bbc8fcff868c0b97 P 49d8879b6e394d1cae1f345f77bd5b83: Deleting consensus metadata
I20260929 19:14:07.149581 29284 catalog_manager.cc:4841] TS 49d8879b6e394d1cae1f345f77bd5b83 (127.9.236.3:40923): tablet cd06ebb394854054bbc8fcff868c0b97 (table test-table [id=1d51b12194ff46ffbe9ae1604c79df4c]) successfully deleted
W20260929 19:14:07.151400 29284 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.236.1:32835 (error 108)
I20260929 19:14:07.151659 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 29495
I20260929 19:14:07.151973 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.151862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:32835 (local address 127.0.0.1:56884)
0929 19:14:07.151909 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:07.151942 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.1:32835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260929 19:14:07.152063 29284 catalog_manager.cc:4584] TS dcf8e641399a4cdda74db6a3369333ac (127.9.236.1:32835): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cd06ebb394854054bbc8fcff868c0b97: Network error: Client connection negotiation failed: client connection to 127.9.236.1:32835: connect: Connection refused (error 111)
I20260929 19:14:07.156451 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 29629
I20260929 19:14:07.161327 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 29267
2026-09-29T19:14:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2088 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-29T19:14:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:07Z Disabled control of system clock
I20260929 19:14:07.189936 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:35779
--webserver_interface=127.9.236.62
--webserver_port=0
--builtin_ntp_servers=127.9.236.20:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:35779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 19:14:07.276559 29805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:07.276818 29805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:07.276837 29805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:07.278090 29805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:07.278134 29805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:07.278146 29805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:07.278156 29805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:07.278167 29805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:07.279546 29805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:35779
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:35779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:07.279822 29805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:07.280210 29805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:07.285302 29811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:07.285305 29810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:07.285319 29813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:07.285390 29805 server_base.cc:1034] running on GCE node
I20260929 19:14:07.285854 29805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:07.286139 29805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:07.287292 29805 hybrid_clock.cc:648] HybridClock initialized: now 1790709247287274 us; error 34 us; skew 500 ppm
I20260929 19:14:07.288621 29805 webserver.cc:460] Webserver started at http://127.9.236.62:39189/ using document root <none> and password file <none>
I20260929 19:14:07.288846 29805 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:07.288901 29805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:07.289021 29805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:07.290153 29805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "276fb25920b84fe99f0b2b28827dd78a"
format_stamp: "Formatted at 2026-09-29 19:14:07 on dist-test-slave-wftq"
server_key: "7a88a916d8c920ee1f0bbdd8d888ec72"
server_key_iv: "aeca601607c87d9f0aa1511d0f60b98a"
server_key_version: "encryptionkey@0"
I20260929 19:14:07.290561 29805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "276fb25920b84fe99f0b2b28827dd78a"
format_stamp: "Formatted at 2026-09-29 19:14:07 on dist-test-slave-wftq"
server_key: "7a88a916d8c920ee1f0bbdd8d888ec72"
server_key_iv: "aeca601607c87d9f0aa1511d0f60b98a"
server_key_version: "encryptionkey@0"
I20260929 19:14:07.291982 29805 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:14:07.292773 29819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:07.292972 29805 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:07.293057 29805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "276fb25920b84fe99f0b2b28827dd78a"
format_stamp: "Formatted at 2026-09-29 19:14:07 on dist-test-slave-wftq"
server_key: "7a88a916d8c920ee1f0bbdd8d888ec72"
server_key_iv: "aeca601607c87d9f0aa1511d0f60b98a"
server_key_version: "encryptionkey@0"
I20260929 19:14:07.293154 29805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:07.309568 29805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:07.310006 29805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:07.310169 29805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:07.315205 29805 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:35779
I20260929 19:14:07.315250 29871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:35779 every 8 connection(s)
I20260929 19:14:07.315635 29805 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:07.316707 29872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:07.319103 29872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a: Bootstrap starting.
I20260929 19:14:07.319664 29872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:07.319926 29872 log.cc:824] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:07.320689 29872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a: No bootstrap required, opened a new log
I20260929 19:14:07.322327 29872 raft_consensus.cc:348] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "276fb25920b84fe99f0b2b28827dd78a" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35779 } }
I20260929 19:14:07.322469 29872 raft_consensus.cc:374] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:07.322501 29872 raft_consensus.cc:729] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 276fb25920b84fe99f0b2b28827dd78a, State: Initialized, Role: FOLLOWER
I20260929 19:14:07.322634 29872 consensus_queue.cc:260] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "276fb25920b84fe99f0b2b28827dd78a" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35779 } }
I20260929 19:14:07.322703 29872 raft_consensus.cc:388] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:07.322733 29872 raft_consensus.cc:482] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:07.322780 29872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:07.323293 29872 raft_consensus.cc:504] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "276fb25920b84fe99f0b2b28827dd78a" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35779 } }
I20260929 19:14:07.323390 29872 leader_election.cc:302] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 276fb25920b84fe99f0b2b28827dd78a; no voters:
I20260929 19:14:07.323558 29872 leader_election.cc:288] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:07.323665 29875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:07.323845 29872 sys_catalog.cc:564] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:07.323917 29875 raft_consensus.cc:686] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a [term 1 LEADER]: Becoming Leader. State: Replica: 276fb25920b84fe99f0b2b28827dd78a, State: Running, Role: LEADER
I20260929 19:14:07.324085 29875 consensus_queue.cc:237] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "276fb25920b84fe99f0b2b28827dd78a" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35779 } }
I20260929 19:14:07.324644 29877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 276fb25920b84fe99f0b2b28827dd78a. Latest consensus state: current_term: 1 leader_uuid: "276fb25920b84fe99f0b2b28827dd78a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "276fb25920b84fe99f0b2b28827dd78a" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35779 } } }
I20260929 19:14:07.324760 29877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:07.324923 29876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "276fb25920b84fe99f0b2b28827dd78a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "276fb25920b84fe99f0b2b28827dd78a" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 35779 } } }
I20260929 19:14:07.325014 29876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:07.325234 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 29805
I20260929 19:14:07.325350 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:07.325555 10160 external_mini_cluster.cc:1428] Setting key 50a2833cf2e30ac4352197f2f2a2c658
I20260929 19:14:07.325459 29884 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:07.326359 29884 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:07.332466 29884 catalog_manager.cc:1347] Generated new cluster ID: 1de6d115ecbb4a7a9290b07300dc58a6
I20260929 19:14:07.332541 29884 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:07.334272 29894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.326215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34398 (local address 127.9.236.62:35779)
0929 19:14:07.326445 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:07.326455 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:07.326488 (+ 33us) server_negotiation.cc:407] Connection header received
0929 19:14:07.326610 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:07.326618 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:07.326693 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:07.326835 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:07.327210 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.328629 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.329963 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.330712 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.330859 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.333295 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:07.333301 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:07.333301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:07.333537 (+ 236us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:07.333588 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:07.333604 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:07.333919 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:07.333953 (+ 34us) server_negotiation.cc:299] Negotiation successful
0929 19:14:07.334052 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":99,"threads_started":1}
I20260929 19:14:07.345336 29884 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:07.346156 29884 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:07.355090 29884 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a: Generated new TSK 0
I20260929 19:14:07.355319 29884 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:07.359792 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.1:0
--local_ip_for_outbound_sockets=127.9.236.1
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:35779
--builtin_ntp_servers=127.9.236.20:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 19:14:07.448165 29896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:07.448411 29896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:07.448441 29896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:07.449685 29896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:07.449726 29896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:07.449759 29896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.1
I20260929 19:14:07.451133 29896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.236.1
--webserver_port=0
--tserver_master_addrs=127.9.236.62:35779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.1
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:07.451373 29896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:07.451674 29896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:07.457002 29902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:07.457168 29901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:07.457135 29896 server_base.cc:1034] running on GCE node
W20260929 19:14:07.457064 29904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:07.457502 29896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:07.457772 29896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:07.458920 29896 hybrid_clock.cc:648] HybridClock initialized: now 1790709247458897 us; error 41 us; skew 500 ppm
I20260929 19:14:07.460227 29896 webserver.cc:460] Webserver started at http://127.9.236.1:34353/ using document root <none> and password file <none>
I20260929 19:14:07.460457 29896 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:07.460506 29896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:07.460635 29896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:07.461783 29896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-0/data/instance:
uuid: "d3bcc7f5710a496489138b8e0ff40018"
format_stamp: "Formatted at 2026-09-29 19:14:07 on dist-test-slave-wftq"
server_key: "4586d232691852e6533d9b2f29742a2d"
server_key_iv: "e2567537ee09601cc74816665bb9ec9d"
server_key_version: "encryptionkey@0"
I20260929 19:14:07.462118 29896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance:
uuid: "d3bcc7f5710a496489138b8e0ff40018"
format_stamp: "Formatted at 2026-09-29 19:14:07 on dist-test-slave-wftq"
server_key: "4586d232691852e6533d9b2f29742a2d"
server_key_iv: "e2567537ee09601cc74816665bb9ec9d"
server_key_version: "encryptionkey@0"
I20260929 19:14:07.463410 29896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:14:07.464318 29910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:07.464576 29896 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:07.464680 29896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-0/wal
uuid: "d3bcc7f5710a496489138b8e0ff40018"
format_stamp: "Formatted at 2026-09-29 19:14:07 on dist-test-slave-wftq"
server_key: "4586d232691852e6533d9b2f29742a2d"
server_key_iv: "e2567537ee09601cc74816665bb9ec9d"
server_key_version: "encryptionkey@0"
I20260929 19:14:07.464751 29896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:07.478456 29896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:07.478852 29896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:07.478986 29896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:07.479209 29896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:14:07.479547 29896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:14:07.479583 29896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:07.479611 29896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:14:07.479627 29896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:07.486053 29896 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.1:38615
I20260929 19:14:07.486173 30023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.1:38615 every 8 connection(s)
I20260929 19:14:07.486420 29896 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-0/data/info.pb
I20260929 19:14:07.492120 29894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.487294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.1:38319 (local address 127.9.236.62:35779)
0929 19:14:07.487408 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:07.487411 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:07.487671 (+ 260us) server_negotiation.cc:407] Connection header received
0929 19:14:07.488299 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:07.488301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:07.488318 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:07.488373 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:07.489146 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.489693 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.490719 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.490925 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.490982 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.491825 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:07.491827 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:07.491827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:07.491835 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:07.491852 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:07.491857 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:07.491912 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:07.491992 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 19:14:07.492024 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 19:14:07.492306 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.487255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:35779 (local address 127.9.236.1:38319)
0929 19:14:07.487515 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:07.487557 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:07.488183 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:07.488460 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:07.488464 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:07.488473 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:07.489031 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:07.489036 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.489776 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.489777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:07.490636 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:07.490638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.490994 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.490995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:07.491124 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.491685 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:14:07.491686 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:07.491718 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:14:07.491745 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:07.491917 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:07.491922 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:07.491925 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 19:14:07.491989 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 19:14:07.492095 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":104,"threads_started":1}
I20260929 19:14:07.492997 30024 heartbeater.cc:347] Connected to a master server at 127.9.236.62:35779
I20260929 19:14:07.493140 30024 heartbeater.cc:464] Registering TS with master...
I20260929 19:14:07.493362 30024 heartbeater.cc:511] Master 127.9.236.62:35779 requested a full tablet report, sending...
I20260929 19:14:07.493956 29836 ts_manager.cc:194] Registered new tserver with Master: d3bcc7f5710a496489138b8e0ff40018 (127.9.236.1:38615)
I20260929 19:14:07.494599 29836 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.236.1:38319
I20260929 19:14:07.495503 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 29896
I20260929 19:14:07.495621 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-0/wal/instance
I20260929 19:14:07.495798 10160 external_mini_cluster.cc:1428] Setting key 6facf818433278cc7917b105035e0007
I20260929 19:14:07.496742 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.2:0
--local_ip_for_outbound_sockets=127.9.236.2
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:35779
--builtin_ntp_servers=127.9.236.20:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 19:14:07.580374 30027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:07.580619 30027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:07.580650 30027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:07.581848 30027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:07.581895 30027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:07.581920 30027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.2
I20260929 19:14:07.583216 30027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.236.2
--webserver_port=0
--tserver_master_addrs=127.9.236.62:35779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.2
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:07.583406 30027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:07.583678 30027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:14:07.588888 30027 server_base.cc:1034] running on GCE node
W20260929 19:14:07.588838 30035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:07.588940 30033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:07.588861 30032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:07.589359 30027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:07.589629 30027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:07.590804 30027 hybrid_clock.cc:648] HybridClock initialized: now 1790709247590786 us; error 37 us; skew 500 ppm
I20260929 19:14:07.592350 30027 webserver.cc:460] Webserver started at http://127.9.236.2:41183/ using document root <none> and password file <none>
I20260929 19:14:07.592626 30027 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:07.592676 30027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:07.592815 30027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:07.594060 30027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-1/data/instance:
uuid: "4f8a485d434741e0b70ed8c158ecedd6"
format_stamp: "Formatted at 2026-09-29 19:14:07 on dist-test-slave-wftq"
server_key: "5517151fe77a19f0276637975974796b"
server_key_iv: "5b464ee3d1cdec132b0ab0ae76943443"
server_key_version: "encryptionkey@0"
I20260929 19:14:07.594468 30027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance:
uuid: "4f8a485d434741e0b70ed8c158ecedd6"
format_stamp: "Formatted at 2026-09-29 19:14:07 on dist-test-slave-wftq"
server_key: "5517151fe77a19f0276637975974796b"
server_key_iv: "5b464ee3d1cdec132b0ab0ae76943443"
server_key_version: "encryptionkey@0"
I20260929 19:14:07.595903 30027 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 19:14:07.596828 30041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:07.597033 30027 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:14:07.597147 30027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-1/wal
uuid: "4f8a485d434741e0b70ed8c158ecedd6"
format_stamp: "Formatted at 2026-09-29 19:14:07 on dist-test-slave-wftq"
server_key: "5517151fe77a19f0276637975974796b"
server_key_iv: "5b464ee3d1cdec132b0ab0ae76943443"
server_key_version: "encryptionkey@0"
I20260929 19:14:07.597230 30027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:07.604797 30027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:07.605129 30027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:07.605259 30027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:07.605479 30027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:14:07.605829 30027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:14:07.605863 30027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:07.605886 30027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:14:07.605911 30027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:07.612672 30027 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.2:43669
I20260929 19:14:07.612731 30154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.2:43669 every 8 connection(s)
I20260929 19:14:07.613085 30027 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-1/data/info.pb
I20260929 19:14:07.618669 29894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.613910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.2:38895 (local address 127.9.236.62:35779)
0929 19:14:07.613979 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:07.613981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:07.614245 (+ 264us) server_negotiation.cc:407] Connection header received
0929 19:14:07.614886 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:07.614888 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:07.614901 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:07.614953 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:07.615680 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.616217 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.617259 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.617450 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.617507 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.618371 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:07.618373 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:07.618373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:07.618378 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:07.618395 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:07.618399 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:07.618442 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:07.618543 (+ 101us) server_negotiation.cc:299] Negotiation successful
0929 19:14:07.618575 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260929 19:14:07.618871 30157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.613897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:35779 (local address 127.9.236.2:38895)
0929 19:14:07.614103 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:07.614141 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:07.614779 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:07.615039 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:07.615043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:07.615054 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:07.615596 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:07.615601 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.616302 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.616303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:07.617186 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:07.617188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.617525 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.617525 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:07.617668 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.618235 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:14:07.618236 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:07.618266 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:14:07.618293 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:07.618447 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:07.618452 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:07.618456 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:14:07.618542 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 19:14:07.618656 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":95,"threads_started":1}
I20260929 19:14:07.619395 30155 heartbeater.cc:347] Connected to a master server at 127.9.236.62:35779
I20260929 19:14:07.619493 30155 heartbeater.cc:464] Registering TS with master...
I20260929 19:14:07.619679 30155 heartbeater.cc:511] Master 127.9.236.62:35779 requested a full tablet report, sending...
I20260929 19:14:07.620139 29836 ts_manager.cc:194] Registered new tserver with Master: 4f8a485d434741e0b70ed8c158ecedd6 (127.9.236.2:43669)
I20260929 19:14:07.620556 29836 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.236.2:38895
I20260929 19:14:07.622256 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 30027
I20260929 19:14:07.622364 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-1/wal/instance
I20260929 19:14:07.622570 10160 external_mini_cluster.cc:1428] Setting key 7f3d3f35cd5033da0d4c1dbd735e5341
I20260929 19:14:07.623503 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.236.3:0
--local_ip_for_outbound_sockets=127.9.236.3
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:35779
--builtin_ntp_servers=127.9.236.20:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 19:14:07.706918 30158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:07.707124 30158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:07.707147 30158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:07.708318 30158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:07.708364 30158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:07.708400 30158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.236.3
I20260929 19:14:07.709787 30158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.236.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.236.3
--webserver_port=0
--tserver_master_addrs=127.9.236.62:35779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.236.3
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:07.710028 30158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:07.710311 30158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:07.715688 30163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:07.715664 30166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:07.715687 30164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:07.715916 30158 server_base.cc:1034] running on GCE node
I20260929 19:14:07.716208 30158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:07.716486 30158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:07.717638 30158 hybrid_clock.cc:648] HybridClock initialized: now 1790709247717616 us; error 41 us; skew 500 ppm
I20260929 19:14:07.719127 30158 webserver.cc:460] Webserver started at http://127.9.236.3:35655/ using document root <none> and password file <none>
I20260929 19:14:07.719388 30158 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:07.719439 30158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:07.719575 30158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:07.720760 30158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-2/data/instance:
uuid: "842542fa731042cb90301eee65b3b8dc"
format_stamp: "Formatted at 2026-09-29 19:14:07 on dist-test-slave-wftq"
server_key: "5957c0157137cc5dd3574eaaf98e06d4"
server_key_iv: "8234aeb773fd475a76ea2e411b2ef70c"
server_key_version: "encryptionkey@0"
I20260929 19:14:07.721172 30158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance:
uuid: "842542fa731042cb90301eee65b3b8dc"
format_stamp: "Formatted at 2026-09-29 19:14:07 on dist-test-slave-wftq"
server_key: "5957c0157137cc5dd3574eaaf98e06d4"
server_key_iv: "8234aeb773fd475a76ea2e411b2ef70c"
server_key_version: "encryptionkey@0"
I20260929 19:14:07.722561 30158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:07.723452 30172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:07.723664 30158 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:07.723744 30158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-2/wal
uuid: "842542fa731042cb90301eee65b3b8dc"
format_stamp: "Formatted at 2026-09-29 19:14:07 on dist-test-slave-wftq"
server_key: "5957c0157137cc5dd3574eaaf98e06d4"
server_key_iv: "8234aeb773fd475a76ea2e411b2ef70c"
server_key_version: "encryptionkey@0"
I20260929 19:14:07.723810 30158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:07.739257 30158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:07.739560 30158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:07.739688 30158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:07.739928 30158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 19:14:07.740273 30158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 19:14:07.740306 30158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:07.740337 30158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 19:14:07.740355 30158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:07.746981 30158 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.3:34819
I20260929 19:14:07.747073 30285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.3:34819 every 8 connection(s)
I20260929 19:14:07.747468 30158 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-2/data/info.pb
I20260929 19:14:07.749042 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 30158
I20260929 19:14:07.749182 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-2/wal/instance
I20260929 19:14:07.749413 10160 external_mini_cluster.cc:1428] Setting key 737dea3f5b1de677f97d6480d3a42cfe
I20260929 19:14:07.753422 29894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.748304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:45415 (local address 127.9.236.62:35779)
0929 19:14:07.748386 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:07.748388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:07.748642 (+ 254us) server_negotiation.cc:407] Connection header received
0929 19:14:07.749505 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:07.749507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:07.749528 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:07.749631 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:07.750527 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.751038 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.752035 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.752246 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.752302 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.753148 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:07.753150 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:07.753150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:07.753155 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:07.753170 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:07.753175 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:07.753227 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:07.753306 (+ 79us) server_negotiation.cc:299] Negotiation successful
0929 19:14:07.753334 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 19:14:07.753609 30288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.748255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:35779 (local address 127.9.236.3:45415)
0929 19:14:07.748487 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:07.748534 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:07.749287 (+ 753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:07.749765 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:07.749771 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:07.749784 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:07.750431 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:07.750438 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.751130 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.751131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:07.751949 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:07.751952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.752322 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.752322 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:07.752458 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.752990 (+ 532us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 19:14:07.752990 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:07.753020 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 19:14:07.753046 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:07.753233 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:07.753238 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:07.753242 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:14:07.753301 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 19:14:07.753404 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":112,"threads_started":1}
I20260929 19:14:07.754115 30286 heartbeater.cc:347] Connected to a master server at 127.9.236.62:35779
I20260929 19:14:07.754201 30286 heartbeater.cc:464] Registering TS with master...
I20260929 19:14:07.754379 30286 heartbeater.cc:511] Master 127.9.236.62:35779 requested a full tablet report, sending...
I20260929 19:14:07.754755 29836 ts_manager.cc:194] Registered new tserver with Master: 842542fa731042cb90301eee65b3b8dc (127.9.236.3:34819)
I20260929 19:14:07.755071 29836 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.236.3:45415
I20260929 19:14:07.761438 10160 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 19:14:07.766000 29894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.763116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34414 (local address 127.9.236.62:35779)
0929 19:14:07.763180 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:07.763182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:07.763295 (+ 113us) server_negotiation.cc:407] Connection header received
0929 19:14:07.763367 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:07.763368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:07.763383 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:07.763428 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:07.763557 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.764118 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.764949 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.765161 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.765221 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.765814 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:07.765815 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:07.765815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:07.765819 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:07.765830 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:07.765834 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:07.765870 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:07.765914 (+ 44us) server_negotiation.cc:299] Negotiation successful
0929 19:14:07.765935 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 19:14:07.767542 29836 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34414:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 19:14:07.769544 29836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 19:14:07.776176 30299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.772097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45420 (local address 127.9.236.3:34819)
0929 19:14:07.772638 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:07.772642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:07.772650 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:14:07.772686 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:07.772688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:07.772704 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:07.772784 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:07.772941 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.773729 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.774964 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.775751 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.775827 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.775920 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:07.776039 (+ 119us) server_negotiation.cc:299] Negotiation successful
0929 19:14:07.776087 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":97,"threads_started":1}
I20260929 19:14:07.776162 30296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.771802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.3:34819 (local address 127.0.0.1:45420)
0929 19:14:07.772475 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:07.772484 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:07.772529 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:07.772806 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:07.772808 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:07.772810 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:07.772859 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:07.772860 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.773822 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.773823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:07.774871 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:07.774873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.775847 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.775849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:07.775927 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.775941 (+ 14us) client_negotiation.cc:769] Sending connection context
0929 19:14:07.776015 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 19:14:07.776081 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"mutex_wait_us":2,"thread_start_us":118,"threads_started":1}
I20260929 19:14:07.776443 30298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.772007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57948 (local address 127.9.236.1:38615)
0929 19:14:07.772235 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:07.772240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:07.772439 (+ 199us) server_negotiation.cc:407] Connection header received
0929 19:14:07.772493 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:07.772498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:07.772515 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:07.772596 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:07.772978 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.774088 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.775299 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.775817 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.775892 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.775992 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:07.776298 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 19:14:07.776359 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":110,"threads_started":1}
I20260929 19:14:07.776597 30297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.772019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54536 (local address 127.9.236.2:43669)
0929 19:14:07.772212 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:07.772218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:07.772229 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:14:07.772268 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:07.772270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:07.772285 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:07.772349 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:07.772532 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.773600 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.775112 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.775765 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.775826 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.775919 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:07.776477 (+ 558us) server_negotiation.cc:299] Negotiation successful
0929 19:14:07.776514 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":79,"threads_started":1}
I20260929 19:14:07.776254 30295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.771802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:38615 (local address 127.0.0.1:57948)
0929 19:14:07.772299 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:07.772311 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:07.772375 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:07.772664 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:07.772665 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:07.772668 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:07.772723 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:07.772726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.774180 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.774181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:07.775225 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:07.775227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.776029 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.776029 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:07.776082 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.776086 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:14:07.776128 (+ 42us) client_negotiation.cc:240] Negotiation successful
0929 19:14:07.776181 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":120,"threads_started":1}
I20260929 19:14:07.777137 30294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.771797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:43669 (local address 127.0.0.1:54536)
0929 19:14:07.772006 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:07.772052 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:07.772143 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:07.772367 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:07.772371 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:07.772373 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:07.772449 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:07.772452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.773691 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.773692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:07.775017 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:07.775020 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.775962 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.775963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:07.776035 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.776042 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 19:14:07.776091 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 19:14:07.776171 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":48,"thread_start_us":89,"threads_started":1}
I20260929 19:14:07.777079 30220 tablet_service.cc:1467] Processing CreateTablet for tablet a5ba9f47c81940bf931484fac46af6a1 (DEFAULT_TABLE table=test-table [id=a5adf781b9cf47fc93eab8359dafa41c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:14:07.777539 30220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5ba9f47c81940bf931484fac46af6a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:07.777484 29958 tablet_service.cc:1467] Processing CreateTablet for tablet a5ba9f47c81940bf931484fac46af6a1 (DEFAULT_TABLE table=test-table [id=a5adf781b9cf47fc93eab8359dafa41c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:14:07.777869 29958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5ba9f47c81940bf931484fac46af6a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:07.777894 30089 tablet_service.cc:1467] Processing CreateTablet for tablet a5ba9f47c81940bf931484fac46af6a1 (DEFAULT_TABLE table=test-table [id=a5adf781b9cf47fc93eab8359dafa41c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 19:14:07.778187 30089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5ba9f47c81940bf931484fac46af6a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:07.780529 30300 tablet_bootstrap.cc:492] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc: Bootstrap starting.
I20260929 19:14:07.781275 30301 tablet_bootstrap.cc:492] T a5ba9f47c81940bf931484fac46af6a1 P 4f8a485d434741e0b70ed8c158ecedd6: Bootstrap starting.
I20260929 19:14:07.781203 30300 tablet_bootstrap.cc:654] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:07.781666 30302 tablet_bootstrap.cc:492] T a5ba9f47c81940bf931484fac46af6a1 P d3bcc7f5710a496489138b8e0ff40018: Bootstrap starting.
I20260929 19:14:07.781744 30300 log.cc:824] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:07.782003 30301 tablet_bootstrap.cc:654] T a5ba9f47c81940bf931484fac46af6a1 P 4f8a485d434741e0b70ed8c158ecedd6: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:07.782305 30301 log.cc:824] T a5ba9f47c81940bf931484fac46af6a1 P 4f8a485d434741e0b70ed8c158ecedd6: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:07.782315 30302 tablet_bootstrap.cc:654] T a5ba9f47c81940bf931484fac46af6a1 P d3bcc7f5710a496489138b8e0ff40018: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:07.782646 30300 tablet_bootstrap.cc:492] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc: No bootstrap required, opened a new log
I20260929 19:14:07.782672 30302 log.cc:824] T a5ba9f47c81940bf931484fac46af6a1 P d3bcc7f5710a496489138b8e0ff40018: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:07.782711 30300 ts_tablet_manager.cc:1397] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260929 19:14:07.783109 30301 tablet_bootstrap.cc:492] T a5ba9f47c81940bf931484fac46af6a1 P 4f8a485d434741e0b70ed8c158ecedd6: No bootstrap required, opened a new log
I20260929 19:14:07.783231 30301 ts_tablet_manager.cc:1397] T a5ba9f47c81940bf931484fac46af6a1 P 4f8a485d434741e0b70ed8c158ecedd6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:14:07.783352 30302 tablet_bootstrap.cc:492] T a5ba9f47c81940bf931484fac46af6a1 P d3bcc7f5710a496489138b8e0ff40018: No bootstrap required, opened a new log
I20260929 19:14:07.783450 30302 ts_tablet_manager.cc:1397] T a5ba9f47c81940bf931484fac46af6a1 P d3bcc7f5710a496489138b8e0ff40018: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 19:14:07.785244 30302 raft_consensus.cc:348] T a5ba9f47c81940bf931484fac46af6a1 P d3bcc7f5710a496489138b8e0ff40018 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842542fa731042cb90301eee65b3b8dc" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34819 } } peers { permanent_uuid: "4f8a485d434741e0b70ed8c158ecedd6" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43669 } } peers { permanent_uuid: "d3bcc7f5710a496489138b8e0ff40018" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 38615 } }
I20260929 19:14:07.785377 30302 raft_consensus.cc:374] T a5ba9f47c81940bf931484fac46af6a1 P d3bcc7f5710a496489138b8e0ff40018 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:07.785396 30302 raft_consensus.cc:729] T a5ba9f47c81940bf931484fac46af6a1 P d3bcc7f5710a496489138b8e0ff40018 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bcc7f5710a496489138b8e0ff40018, State: Initialized, Role: FOLLOWER
I20260929 19:14:07.785488 30302 consensus_queue.cc:260] T a5ba9f47c81940bf931484fac46af6a1 P d3bcc7f5710a496489138b8e0ff40018 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842542fa731042cb90301eee65b3b8dc" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34819 } } peers { permanent_uuid: "4f8a485d434741e0b70ed8c158ecedd6" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43669 } } peers { permanent_uuid: "d3bcc7f5710a496489138b8e0ff40018" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 38615 } }
I20260929 19:14:07.785715 30302 ts_tablet_manager.cc:1428] T a5ba9f47c81940bf931484fac46af6a1 P d3bcc7f5710a496489138b8e0ff40018: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 19:14:07.785769 30024 heartbeater.cc:503] Master 127.9.236.62:35779 was elected leader, sending a full tablet report...
I20260929 19:14:07.785586 30300 raft_consensus.cc:348] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842542fa731042cb90301eee65b3b8dc" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34819 } } peers { permanent_uuid: "4f8a485d434741e0b70ed8c158ecedd6" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43669 } } peers { permanent_uuid: "d3bcc7f5710a496489138b8e0ff40018" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 38615 } }
I20260929 19:14:07.786118 30300 raft_consensus.cc:374] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:07.786166 30300 raft_consensus.cc:729] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842542fa731042cb90301eee65b3b8dc, State: Initialized, Role: FOLLOWER
I20260929 19:14:07.786301 30300 consensus_queue.cc:260] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842542fa731042cb90301eee65b3b8dc" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34819 } } peers { permanent_uuid: "4f8a485d434741e0b70ed8c158ecedd6" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43669 } } peers { permanent_uuid: "d3bcc7f5710a496489138b8e0ff40018" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 38615 } }
I20260929 19:14:07.786419 30301 raft_consensus.cc:348] T a5ba9f47c81940bf931484fac46af6a1 P 4f8a485d434741e0b70ed8c158ecedd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842542fa731042cb90301eee65b3b8dc" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34819 } } peers { permanent_uuid: "4f8a485d434741e0b70ed8c158ecedd6" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43669 } } peers { permanent_uuid: "d3bcc7f5710a496489138b8e0ff40018" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 38615 } }
I20260929 19:14:07.786572 30301 raft_consensus.cc:374] T a5ba9f47c81940bf931484fac46af6a1 P 4f8a485d434741e0b70ed8c158ecedd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:07.786594 30300 ts_tablet_manager.cc:1428] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260929 19:14:07.786603 30301 raft_consensus.cc:729] T a5ba9f47c81940bf931484fac46af6a1 P 4f8a485d434741e0b70ed8c158ecedd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f8a485d434741e0b70ed8c158ecedd6, State: Initialized, Role: FOLLOWER
I20260929 19:14:07.786697 30301 consensus_queue.cc:260] T a5ba9f47c81940bf931484fac46af6a1 P 4f8a485d434741e0b70ed8c158ecedd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842542fa731042cb90301eee65b3b8dc" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34819 } } peers { permanent_uuid: "4f8a485d434741e0b70ed8c158ecedd6" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43669 } } peers { permanent_uuid: "d3bcc7f5710a496489138b8e0ff40018" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 38615 } }
I20260929 19:14:07.786790 30286 heartbeater.cc:503] Master 127.9.236.62:35779 was elected leader, sending a full tablet report...
I20260929 19:14:07.786945 30301 ts_tablet_manager.cc:1428] T a5ba9f47c81940bf931484fac46af6a1 P 4f8a485d434741e0b70ed8c158ecedd6: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260929 19:14:07.786962 30155 heartbeater.cc:503] Master 127.9.236.62:35779 was elected leader, sending a full tablet report...
I20260929 19:14:07.790557 30307 raft_consensus.cc:482] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 19:14:07.790658 30307 raft_consensus.cc:504] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842542fa731042cb90301eee65b3b8dc" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34819 } } peers { permanent_uuid: "4f8a485d434741e0b70ed8c158ecedd6" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43669 } } peers { permanent_uuid: "d3bcc7f5710a496489138b8e0ff40018" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 38615 } }
I20260929 19:14:07.790971 30307 leader_election.cc:288] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f8a485d434741e0b70ed8c158ecedd6 (127.9.236.2:43669), d3bcc7f5710a496489138b8e0ff40018 (127.9.236.1:38615)
I20260929 19:14:07.794137 30297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.791380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:40875 (local address 127.9.236.2:43669)
0929 19:14:07.791443 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:07.791445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:07.791453 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:14:07.791477 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:07.791478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:07.791486 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:07.791537 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:07.791740 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.792190 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.793431 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.793770 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.793825 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.793833 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:07.794030 (+ 197us) server_negotiation.cc:299] Negotiation successful
0929 19:14:07.794064 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 19:14:07.794137 30298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.791293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.236.3:39793 (local address 127.9.236.1:38615)
0929 19:14:07.791374 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:07.791377 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:07.791394 (+ 17us) server_negotiation.cc:407] Connection header received
0929 19:14:07.791422 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:07.791423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:07.791435 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:07.791494 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 19:14:07.791673 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.792202 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.793405 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.793794 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.793861 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.793874 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:07.794033 (+ 159us) server_negotiation.cc:299] Negotiation successful
0929 19:14:07.794073 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 19:14:07.794163 30288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.791136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:38615 (local address 127.9.236.3:39793)
0929 19:14:07.791180 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:07.791190 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:07.791229 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:07.791500 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:07.791501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:07.791502 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:07.791569 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:07.791571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.792283 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.792284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:07.793320 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:07.793323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.793878 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.793879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:07.793962 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.793966 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:14:07.794016 (+ 50us) client_negotiation.cc:240] Negotiation successful
0929 19:14:07.794067 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":53}
I20260929 19:14:07.794181 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.791167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.2:43669 (local address 127.9.236.3:40875)
0929 19:14:07.791358 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:07.791368 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:07.791408 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:07.791563 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:07.791564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:07.791567 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 19:14:07.791665 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:07.791667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.792304 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.792305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:07.793347 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:07.793350 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.793845 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.793845 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:07.793960 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.793964 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 19:14:07.794009 (+ 45us) client_negotiation.cc:240] Negotiation successful
0929 19:14:07.794056 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":1,"thread_start_us":106,"threads_started":1}
I20260929 19:14:07.794557 30109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ba9f47c81940bf931484fac46af6a1" candidate_uuid: "842542fa731042cb90301eee65b3b8dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f8a485d434741e0b70ed8c158ecedd6" is_pre_election: true
I20260929 19:14:07.794569 29978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ba9f47c81940bf931484fac46af6a1" candidate_uuid: "842542fa731042cb90301eee65b3b8dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bcc7f5710a496489138b8e0ff40018" is_pre_election: true
I20260929 19:14:07.794685 29978 raft_consensus.cc:2454] T a5ba9f47c81940bf931484fac46af6a1 P d3bcc7f5710a496489138b8e0ff40018 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842542fa731042cb90301eee65b3b8dc in term 0.
I20260929 19:14:07.794688 30109 raft_consensus.cc:2454] T a5ba9f47c81940bf931484fac46af6a1 P 4f8a485d434741e0b70ed8c158ecedd6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842542fa731042cb90301eee65b3b8dc in term 0.
I20260929 19:14:07.794888 30173 leader_election.cc:302] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842542fa731042cb90301eee65b3b8dc, d3bcc7f5710a496489138b8e0ff40018; no voters:
I20260929 19:14:07.795037 30307 raft_consensus.cc:2781] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 19:14:07.795074 30307 raft_consensus.cc:482] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 19:14:07.795095 30307 raft_consensus.cc:3037] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:07.795778 30307 raft_consensus.cc:504] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842542fa731042cb90301eee65b3b8dc" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34819 } } peers { permanent_uuid: "4f8a485d434741e0b70ed8c158ecedd6" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43669 } } peers { permanent_uuid: "d3bcc7f5710a496489138b8e0ff40018" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 38615 } }
I20260929 19:14:07.795953 30307 leader_election.cc:288] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc [CANDIDATE]: Term 1 election: Requested vote from peers 4f8a485d434741e0b70ed8c158ecedd6 (127.9.236.2:43669), d3bcc7f5710a496489138b8e0ff40018 (127.9.236.1:38615)
I20260929 19:14:07.796048 29978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ba9f47c81940bf931484fac46af6a1" candidate_uuid: "842542fa731042cb90301eee65b3b8dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3bcc7f5710a496489138b8e0ff40018"
I20260929 19:14:07.796111 29978 raft_consensus.cc:3037] T a5ba9f47c81940bf931484fac46af6a1 P d3bcc7f5710a496489138b8e0ff40018 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:07.796106 30109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ba9f47c81940bf931484fac46af6a1" candidate_uuid: "842542fa731042cb90301eee65b3b8dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f8a485d434741e0b70ed8c158ecedd6"
I20260929 19:14:07.796177 30109 raft_consensus.cc:3037] T a5ba9f47c81940bf931484fac46af6a1 P 4f8a485d434741e0b70ed8c158ecedd6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:07.796821 30109 raft_consensus.cc:2454] T a5ba9f47c81940bf931484fac46af6a1 P 4f8a485d434741e0b70ed8c158ecedd6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842542fa731042cb90301eee65b3b8dc in term 1.
I20260929 19:14:07.796978 29978 raft_consensus.cc:2454] T a5ba9f47c81940bf931484fac46af6a1 P d3bcc7f5710a496489138b8e0ff40018 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842542fa731042cb90301eee65b3b8dc in term 1.
I20260929 19:14:07.797073 30175 leader_election.cc:302] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f8a485d434741e0b70ed8c158ecedd6, 842542fa731042cb90301eee65b3b8dc; no voters:
I20260929 19:14:07.797200 30307 raft_consensus.cc:2781] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:07.797351 30307 raft_consensus.cc:686] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc [term 1 LEADER]: Becoming Leader. State: Replica: 842542fa731042cb90301eee65b3b8dc, State: Running, Role: LEADER
I20260929 19:14:07.797451 30307 consensus_queue.cc:237] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842542fa731042cb90301eee65b3b8dc" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34819 } } peers { permanent_uuid: "4f8a485d434741e0b70ed8c158ecedd6" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43669 } } peers { permanent_uuid: "d3bcc7f5710a496489138b8e0ff40018" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 38615 } }
I20260929 19:14:07.798277 29835 catalog_manager.cc:5495] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 842542fa731042cb90301eee65b3b8dc (127.9.236.3). New cstate: current_term: 1 leader_uuid: "842542fa731042cb90301eee65b3b8dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "842542fa731042cb90301eee65b3b8dc" member_type: VOTER last_known_addr { host: "127.9.236.3" port: 34819 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f8a485d434741e0b70ed8c158ecedd6" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3bcc7f5710a496489138b8e0ff40018" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 38615 } health_report { overall_health: UNKNOWN } } }
I20260929 19:14:07.803443 30299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.801020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45436 (local address 127.9.236.3:34819)
0929 19:14:07.801082 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:07.801084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:07.801094 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:14:07.801203 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:07.801204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:07.801220 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:07.801289 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:07.801416 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.801855 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.802712 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:07.802921 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:07.802976 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:07.803115 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:07.803221 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:07.803291 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:07.803312 (+ 21us) server_negotiation.cc:299] Negotiation successful
0929 19:14:07.803335 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 19:14:07.805238 29978 raft_consensus.cc:1261] T a5ba9f47c81940bf931484fac46af6a1 P d3bcc7f5710a496489138b8e0ff40018 [term 1 FOLLOWER]: Refusing update from remote peer 842542fa731042cb90301eee65b3b8dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:14:07.805299 30109 raft_consensus.cc:1261] T a5ba9f47c81940bf931484fac46af6a1 P 4f8a485d434741e0b70ed8c158ecedd6 [term 1 FOLLOWER]: Refusing update from remote peer 842542fa731042cb90301eee65b3b8dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:14:07.805677 30307 consensus_queue.cc:1035] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3bcc7f5710a496489138b8e0ff40018" member_type: VOTER last_known_addr { host: "127.9.236.1" port: 38615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:14:07.805821 30310 consensus_queue.cc:1035] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f8a485d434741e0b70ed8c158ecedd6" member_type: VOTER last_known_addr { host: "127.9.236.2" port: 43669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:14:07.811339 29836 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:14:07.811444 29836 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 19:14:07.812234 29836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 276fb25920b84fe99f0b2b28827dd78a: Sending DeleteTablet for 3 replicas of tablet a5ba9f47c81940bf931484fac46af6a1
I20260929 19:14:07.812558 30220 tablet_service.cc:1514] Processing DeleteTablet for tablet a5ba9f47c81940bf931484fac46af6a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:14:07 UTC) from {username='slave'} at 127.0.0.1:45420
I20260929 19:14:07.812741 30089 tablet_service.cc:1514] Processing DeleteTablet for tablet a5ba9f47c81940bf931484fac46af6a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:14:07 UTC) from {username='slave'} at 127.0.0.1:54536
I20260929 19:14:07.812943 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 29896
I20260929 19:14:07.812968 29958 tablet_service.cc:1514] Processing DeleteTablet for tablet a5ba9f47c81940bf931484fac46af6a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 19:14:07 UTC) from {username='slave'} at 127.0.0.1:57948
I20260929 19:14:07.815243 30324 tablet_replica.cc:330] T a5ba9f47c81940bf931484fac46af6a1 P 4f8a485d434741e0b70ed8c158ecedd6: stopping tablet replica
I20260929 19:14:07.815382 30324 raft_consensus.cc:2229] T a5ba9f47c81940bf931484fac46af6a1 P 4f8a485d434741e0b70ed8c158ecedd6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 19:14:07.815495 30324 raft_consensus.cc:2258] T a5ba9f47c81940bf931484fac46af6a1 P 4f8a485d434741e0b70ed8c158ecedd6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:14:07.815971 30324 ts_tablet_manager.cc:1905] T a5ba9f47c81940bf931484fac46af6a1 P 4f8a485d434741e0b70ed8c158ecedd6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:14:07.817350 30323 tablet_replica.cc:330] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc: stopping tablet replica
I20260929 19:14:07.817497 30323 raft_consensus.cc:2229] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc [term 1 LEADER]: Raft consensus shutting down.
I20260929 19:14:07.817657 30323 raft_consensus.cc:2258] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 19:14:07.817941 30324 ts_tablet_manager.cc:1918] T a5ba9f47c81940bf931484fac46af6a1 P 4f8a485d434741e0b70ed8c158ecedd6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:14:07.818017 30324 log.cc:1196] T a5ba9f47c81940bf931484fac46af6a1 P 4f8a485d434741e0b70ed8c158ecedd6: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-1/wal/wals/a5ba9f47c81940bf931484fac46af6a1
I20260929 19:14:07.818252 30324 ts_tablet_manager.cc:1939] T a5ba9f47c81940bf931484fac46af6a1 P 4f8a485d434741e0b70ed8c158ecedd6: Deleting consensus metadata
I20260929 19:14:07.818388 30323 ts_tablet_manager.cc:1905] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 19:14:07.818660 29822 catalog_manager.cc:4841] TS 4f8a485d434741e0b70ed8c158ecedd6 (127.9.236.2:43669): tablet a5ba9f47c81940bf931484fac46af6a1 (table test-table [id=a5adf781b9cf47fc93eab8359dafa41c]) successfully deleted
W20260929 19:14:07.819162 29820 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.236.1:38615 (error 108)
I20260929 19:14:07.819458 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 30027
I20260929 19:14:07.820331 30323 ts_tablet_manager.cc:1918] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 19:14:07.820477 30323 log.cc:1196] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc: Deleting WAL directory at /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790709008521303-10160-0/minicluster-data/ts-2/wal/wals/a5ba9f47c81940bf931484fac46af6a1
I20260929 19:14:07.820775 30323 ts_tablet_manager.cc:1939] T a5ba9f47c81940bf931484fac46af6a1 P 842542fa731042cb90301eee65b3b8dc: Deleting consensus metadata
I20260929 19:14:07.821282 29821 catalog_manager.cc:4841] TS 842542fa731042cb90301eee65b3b8dc (127.9.236.3:34819): tablet a5ba9f47c81940bf931484fac46af6a1 (table test-table [id=a5adf781b9cf47fc93eab8359dafa41c]) successfully deleted
I20260929 19:14:07.824983 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 30158
I20260929 19:14:07.825042 30294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:07.824888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.1:38615 (local address 127.0.0.1:57950)
0929 19:14:07.824954 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:07.825002 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.1:38615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 19:14:07.826401 29820 catalog_manager.cc:4584] TS d3bcc7f5710a496489138b8e0ff40018 (127.9.236.1:38615): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a5ba9f47c81940bf931484fac46af6a1: Network error: Client connection negotiation failed: client connection to 127.9.236.1:38615: connect: Connection refused (error 111)
I20260929 19:14:07.830351 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 29805
2026-09-29T19:14:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (668 ms)
[----------] 6 tests from EncryptionPolicyTest (8091 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:07 dist-test-slave-wftq krb5kdc[30328](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:07 dist-test-slave-wftq krb5kdc[30328](info): set up 2 sockets
Sep 29 19:14:07 dist-test-slave-wftq krb5kdc[30328](info): commencing operation
krb5kdc: starting...
W20260929 19:14:08.780933 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:08 dist-test-slave-wftq krb5kdc[30328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:14:08.900913 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39721
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:08.986590 30341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:08.986836 30341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:08.986860 30341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:08.988008 30341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:08.988046 30341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:08.988060 30341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:08.988070 30341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:08.988081 30341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:08.989619 30341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39721
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:08.989861 30341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:08.990068 30341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:14:08.995345 30341 server_base.cc:1034] running on GCE node
W20260929 19:14:08.995261 30349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:08.995235 30347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:08.995250 30346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:08.995801 30341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:08.996135 30341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:08.997282 30341 hybrid_clock.cc:648] HybridClock initialized: now 1790709248997249 us; error 49 us; skew 500 ppm
Sep 29 19:14:08 dist-test-slave-wftq krb5kdc[30328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709248, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:09.000063 30341 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:14:09.000612 30341 webserver.cc:460] Webserver started at http://127.0.0.1:45947/ using document root <none> and password file <none>
I20260929 19:14:09.000859 30341 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:09.000908 30341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:09.001044 30341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:09.002250 30341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "09938b686dde40ffa07c8c90d1608116"
format_stamp: "Formatted at 2026-09-29 19:14:08 on dist-test-slave-wftq"
server_key: "86dc172012bfd264a8c4170e239a5c44"
server_key_iv: "9577b55272fdf5b2080f6f619e75d714"
server_key_version: "encryptionkey@0"
I20260929 19:14:09.002628 30341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "09938b686dde40ffa07c8c90d1608116"
format_stamp: "Formatted at 2026-09-29 19:14:08 on dist-test-slave-wftq"
server_key: "86dc172012bfd264a8c4170e239a5c44"
server_key_iv: "9577b55272fdf5b2080f6f619e75d714"
server_key_version: "encryptionkey@0"
I20260929 19:14:09.004027 30341 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:14:09.004881 30356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:09.005087 30341 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:09.005211 30341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "09938b686dde40ffa07c8c90d1608116"
format_stamp: "Formatted at 2026-09-29 19:14:08 on dist-test-slave-wftq"
server_key: "86dc172012bfd264a8c4170e239a5c44"
server_key_iv: "9577b55272fdf5b2080f6f619e75d714"
server_key_version: "encryptionkey@0"
I20260929 19:14:09.005285 30341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:09.016156 30341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:09.018262 30341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:09.018402 30341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:09.023016 30341 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39721
I20260929 19:14:09.023078 30408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39721 every 8 connection(s)
I20260929 19:14:09.023435 30341 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:09.024374 30409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:09.025951 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 30341
I20260929 19:14:09.026070 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:09.026268 10160 external_mini_cluster.cc:1428] Setting key acf63d0a3895f84e82ee3d2409b0766e
I20260929 19:14:09.026927 30409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116: Bootstrap starting.
I20260929 19:14:09.027510 30409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:09.027787 30409 log.cc:824] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:09.028615 30409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116: No bootstrap required, opened a new log
I20260929 19:14:09.031030 30409 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09938b686dde40ffa07c8c90d1608116" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39721 } }
I20260929 19:14:09.031207 30409 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:09.031241 30409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09938b686dde40ffa07c8c90d1608116, State: Initialized, Role: FOLLOWER
I20260929 19:14:09.031392 30409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09938b686dde40ffa07c8c90d1608116" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39721 } }
I20260929 19:14:09.031471 30409 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:09.031503 30409 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:09.031553 30409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:09.032231 30409 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09938b686dde40ffa07c8c90d1608116" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39721 } }
I20260929 19:14:09.032341 30409 leader_election.cc:302] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09938b686dde40ffa07c8c90d1608116; no voters:
I20260929 19:14:09.032544 30409 leader_election.cc:288] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:09.032614 30414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:09.032757 30414 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116 [term 1 LEADER]: Becoming Leader. State: Replica: 09938b686dde40ffa07c8c90d1608116, State: Running, Role: LEADER
I20260929 19:14:09.032862 30409 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:09.032847 30414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09938b686dde40ffa07c8c90d1608116" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39721 } }
I20260929 19:14:09.033325 30415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09938b686dde40ffa07c8c90d1608116" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09938b686dde40ffa07c8c90d1608116" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39721 } } }
I20260929 19:14:09.033453 30415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:09.033310 30414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09938b686dde40ffa07c8c90d1608116. Latest consensus state: current_term: 1 leader_uuid: "09938b686dde40ffa07c8c90d1608116" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09938b686dde40ffa07c8c90d1608116" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39721 } } }
I20260929 19:14:09.033667 30414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116 [sys.catalog]: This master's current role is: LEADER
Sep 29 19:14:09 dist-test-slave-wftq krb5kdc[30328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 19:14:09.033797 30421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:09.034374 30421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:09.036298 30421 catalog_manager.cc:1347] Generated new cluster ID: bdc54212111548a0a8127e94a9b68110
I20260929 19:14:09.036361 30421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:09.037878 30412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:09.026914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44505 (local address 127.0.0.1:39721)
0929 19:14:09.027126 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:09.027135 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:09.027167 (+ 32us) server_negotiation.cc:407] Connection header received
0929 19:14:09.027792 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:09.027799 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:09.027871 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:09.028034 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:09.028758 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:09.029746 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:09.030960 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:09.031469 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:09.031585 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:09.034334 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:09.034338 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:09.034339 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:09.034352 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:09.036304 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:09.036681 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:09.036684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:09.036684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:09.036786 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:09.036982 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:09.036983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:09.036983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:09.037296 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:09.037432 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:09.037562 (+ 130us) server_negotiation.cc:299] Negotiation successful
0929 19:14:09.037659 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":87,"threads_started":1}
I20260929 19:14:09.038774 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:14:09.045868 30412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:09.040150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37641 (local address 127.0.0.1:39721)
0929 19:14:09.040213 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:09.040215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:09.040250 (+ 35us) server_negotiation.cc:407] Connection header received
0929 19:14:09.040304 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:09.040305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:09.040314 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:09.040356 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:09.040828 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:09.041633 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:09.042301 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:09.042549 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:09.042601 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:09.043629 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:09.043630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:09.043630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:09.043637 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:09.044860 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:09.045185 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:09.045186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:09.045186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:09.045235 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:09.045415 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:09.045416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:09.045416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:09.045518 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:09.045610 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:09.045727 (+ 117us) server_negotiation.cc:299] Negotiation successful
0929 19:14:09.045783 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 19:14:09.086036 30421 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:09.086841 30421 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:09.095361 30421 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09938b686dde40ffa07c8c90d1608116: Generated new TSK 0
I20260929 19:14:09.095577 30421 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:09.097605 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 30341
W20260929 19:14:09.101861 30326 connection.cc:535] client connection to 127.0.0.1:39721 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39721): Connection reset by peer (error 104)
2026-09-29T19:14:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1266 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:09 dist-test-slave-wftq krb5kdc[30440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:09 dist-test-slave-wftq krb5kdc[30440](info): set up 2 sockets
Sep 29 19:14:09 dist-test-slave-wftq krb5kdc[30440](info): commencing operation
krb5kdc: starting...
W20260929 19:14:10.045207 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:10 dist-test-slave-wftq krb5kdc[30440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:14:10.166275 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42105
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:10.248267 30453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:10.248466 30453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:10.248495 30453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:10.249643 30453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:10.249686 30453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:10.249706 30453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:10.249720 30453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:10.249737 30453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:10.251163 30453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42105
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:10.251399 30453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:10.251588 30453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:10.256770 30458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:10.256857 30461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:10.256767 30459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:10.256879 30453 server_base.cc:1034] running on GCE node
I20260929 19:14:10.257284 30453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:10.257551 30453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:10.258692 30453 hybrid_clock.cc:648] HybridClock initialized: now 1790709250258670 us; error 40 us; skew 500 ppm
Sep 29 19:14:10 dist-test-slave-wftq krb5kdc[30440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709250, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:10.266305 30453 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:14:10.266796 30453 webserver.cc:460] Webserver started at http://127.0.0.1:36729/ using document root <none> and password file <none>
I20260929 19:14:10.267006 30453 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:10.267055 30453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:10.267168 30453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:10.268220 30453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "7fe9a43dc1944270a34b13322339cfbc"
format_stamp: "Formatted at 2026-09-29 19:14:10 on dist-test-slave-wftq"
server_key: "e5859f26cc457d757b100af792825818"
server_key_iv: "f77410d01753e8fcc7905545c0cbee67"
server_key_version: "encryptionkey@0"
I20260929 19:14:10.268541 30453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "7fe9a43dc1944270a34b13322339cfbc"
format_stamp: "Formatted at 2026-09-29 19:14:10 on dist-test-slave-wftq"
server_key: "e5859f26cc457d757b100af792825818"
server_key_iv: "f77410d01753e8fcc7905545c0cbee67"
server_key_version: "encryptionkey@0"
I20260929 19:14:10.269878 30453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260929 19:14:10.270589 30468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:10.270788 30453 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:10.270879 30453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "7fe9a43dc1944270a34b13322339cfbc"
format_stamp: "Formatted at 2026-09-29 19:14:10 on dist-test-slave-wftq"
server_key: "e5859f26cc457d757b100af792825818"
server_key_iv: "f77410d01753e8fcc7905545c0cbee67"
server_key_version: "encryptionkey@0"
I20260929 19:14:10.270941 30453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:10.297258 30453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:10.299273 30453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:10.299407 30453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:10.303571 30453 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42105
I20260929 19:14:10.303644 30520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42105 every 8 connection(s)
I20260929 19:14:10.303980 30453 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:10.304947 30521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:10.307524 30521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc: Bootstrap starting.
I20260929 19:14:10.308111 30521 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:10.308367 30521 log.cc:824] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:10.309129 30521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc: No bootstrap required, opened a new log
I20260929 19:14:10.310741 30521 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe9a43dc1944270a34b13322339cfbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42105 } }
I20260929 19:14:10.310894 30521 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:10.310917 30521 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fe9a43dc1944270a34b13322339cfbc, State: Initialized, Role: FOLLOWER
I20260929 19:14:10.311038 30521 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe9a43dc1944270a34b13322339cfbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42105 } }
I20260929 19:14:10.311089 30521 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:10.311117 30521 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:10.311151 30521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:10.311651 30521 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe9a43dc1944270a34b13322339cfbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42105 } }
I20260929 19:14:10.311741 30521 leader_election.cc:302] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fe9a43dc1944270a34b13322339cfbc; no voters:
I20260929 19:14:10.311929 30521 leader_election.cc:288] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:10.311961 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 30453
I20260929 19:14:10.311980 30524 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:10.312075 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:10.312162 30524 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc [term 1 LEADER]: Becoming Leader. State: Replica: 7fe9a43dc1944270a34b13322339cfbc, State: Running, Role: LEADER
I20260929 19:14:10.312222 30521 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:10.312288 10160 external_mini_cluster.cc:1428] Setting key cfafb50ce66f575f513a20ddb8a87232
I20260929 19:14:10.312278 30524 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe9a43dc1944270a34b13322339cfbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42105 } }
I20260929 19:14:10.312696 30524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fe9a43dc1944270a34b13322339cfbc. Latest consensus state: current_term: 1 leader_uuid: "7fe9a43dc1944270a34b13322339cfbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe9a43dc1944270a34b13322339cfbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42105 } } }
I20260929 19:14:10.312775 30524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:10.312875 30526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7fe9a43dc1944270a34b13322339cfbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe9a43dc1944270a34b13322339cfbc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42105 } } }
I20260929 19:14:10.312932 30526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:10.313910 30532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:10.314592 30532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:10.316462 30532 catalog_manager.cc:1347] Generated new cluster ID: d395f0ff566244bba6adcf5903d368c7
I20260929 19:14:10.316509 30532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:10.318810 30535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:10.312982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40553 (local address 127.0.0.1:42105)
0929 19:14:10.313205 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:10.313215 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:10.313336 (+ 121us) server_negotiation.cc:407] Connection header received
0929 19:14:10.313525 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:10.313532 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:10.313603 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:10.313784 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:10.314035 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:10.315358 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:10.316432 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:10.317050 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:10.317584 (+ 534us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:10.318030 (+ 446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:10.318034 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:10.318034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:10.318234 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:10.318272 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:10.318287 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:10.318523 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:10.318564 (+ 41us) server_negotiation.cc:299] Negotiation successful
0929 19:14:10.318591 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":93,"threads_started":1}
I20260929 19:14:10.319721 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:14:10.328742 30535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:10.321272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59215 (local address 127.0.0.1:42105)
0929 19:14:10.321320 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:10.321323 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:10.325270 (+ 3947us) server_negotiation.cc:407] Connection header received
0929 19:14:10.325321 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:10.325322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:10.325334 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:10.325392 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:10.325538 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:10.326304 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:10.327343 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:10.327670 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:10.327746 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:10.328481 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:10.328482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:10.328483 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:10.328487 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:10.328503 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:10.328508 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:10.328567 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:10.328647 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 19:14:10.328660 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 19:14:10.337067 30532 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:10.337641 30532 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:10.342237 30532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7fe9a43dc1944270a34b13322339cfbc: Generated new TSK 0
I20260929 19:14:10.342434 30532 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:10.345963 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 30453
W20260929 19:14:10.349874 30438 connection.cc:535] client connection to 127.0.0.1:42105 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42105): Connection reset by peer (error 104)
2026-09-29T19:14:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1247 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:10 dist-test-slave-wftq krb5kdc[30552](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:10 dist-test-slave-wftq krb5kdc[30552](info): set up 2 sockets
Sep 29 19:14:10 dist-test-slave-wftq krb5kdc[30552](info): commencing operation
krb5kdc: starting...
W20260929 19:14:11.308980 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.945s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:11 dist-test-slave-wftq krb5kdc[30552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:14:11.429741 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34383
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:11.506994 30565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:11.507189 30565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:11.507210 30565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:11.508278 30565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:11.508315 30565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:11.508328 30565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:11.508339 30565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:11.508350 30565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:11.509750 30565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34383
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:11.509940 30565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:11.510098 30565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:11.515056 30571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:11.515071 30565 server_base.cc:1034] running on GCE node
W20260929 19:14:11.515034 30570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:11.515033 30573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:11.515446 30565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:11.515715 30565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:11.516875 30565 hybrid_clock.cc:648] HybridClock initialized: now 1790709251516849 us; error 47 us; skew 500 ppm
Sep 29 19:14:11 dist-test-slave-wftq krb5kdc[30552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709251, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:11.519284 30565 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:14:11.519722 30565 webserver.cc:460] Webserver started at http://127.0.0.1:32783/ using document root <none> and password file <none>
I20260929 19:14:11.519945 30565 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:11.520002 30565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:11.520113 30565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:11.521095 30565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "d00bb9d0ca4949d7b0aae57fd6beaf02"
format_stamp: "Formatted at 2026-09-29 19:14:11 on dist-test-slave-wftq"
server_key: "e6b2db77d78998a0a7512a9c5dbba439"
server_key_iv: "38080a00e9d9e40e2287a30ce67291c5"
server_key_version: "encryptionkey@0"
I20260929 19:14:11.521468 30565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "d00bb9d0ca4949d7b0aae57fd6beaf02"
format_stamp: "Formatted at 2026-09-29 19:14:11 on dist-test-slave-wftq"
server_key: "e6b2db77d78998a0a7512a9c5dbba439"
server_key_iv: "38080a00e9d9e40e2287a30ce67291c5"
server_key_version: "encryptionkey@0"
I20260929 19:14:11.522727 30565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:11.523452 30580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:11.523648 30565 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:11.523734 30565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "d00bb9d0ca4949d7b0aae57fd6beaf02"
format_stamp: "Formatted at 2026-09-29 19:14:11 on dist-test-slave-wftq"
server_key: "e6b2db77d78998a0a7512a9c5dbba439"
server_key_iv: "38080a00e9d9e40e2287a30ce67291c5"
server_key_version: "encryptionkey@0"
I20260929 19:14:11.523810 30565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:11.526978 30565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:11.528680 30565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:11.528816 30565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:11.532804 30565 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34383
I20260929 19:14:11.532851 30632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34383 every 8 connection(s)
I20260929 19:14:11.533190 30565 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:11.534281 30633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:11.534406 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 30565
I20260929 19:14:11.534508 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:11.534684 10160 external_mini_cluster.cc:1428] Setting key cc98f15dfda3b28a8d7b00b677918e13
I20260929 19:14:11.537256 30636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:11.535217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39361 (local address 127.0.0.1:34383)
0929 19:14:11.535427 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:11.535438 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:11.535474 (+ 36us) server_negotiation.cc:407] Connection header received
0929 19:14:11.535985 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:11.535996 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:11.536034 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:11.536211 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:11.536785 (+ 574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:11.536789 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:11.536789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:11.536794 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:11.536840 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:11.536854 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:11.536940 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:11.536985 (+ 45us) server_negotiation.cc:299] Negotiation successful
0929 19:14:11.537027 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":93,"threads_started":1}
I20260929 19:14:11.537854 30633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02: Bootstrap starting.
I20260929 19:14:11.538419 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:14:11.538636 30633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:11.538904 30633 log.cc:824] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:11.539601 30633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02: No bootstrap required, opened a new log
I20260929 19:14:11.540925 30636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:11.539899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55035 (local address 127.0.0.1:34383)
0929 19:14:11.539945 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:11.539947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:11.540015 (+ 68us) server_negotiation.cc:407] Connection header received
0929 19:14:11.540085 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:11.540086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:11.540091 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:11.540139 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:11.540719 (+ 580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:11.540720 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:11.540721 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:11.540724 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:11.540740 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:11.540744 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:11.540795 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:11.540862 (+ 67us) server_negotiation.cc:299] Negotiation successful
0929 19:14:11.540873 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 19:14:11.542050 30633 raft_consensus.cc:348] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00bb9d0ca4949d7b0aae57fd6beaf02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34383 } }
I20260929 19:14:11.542198 30633 raft_consensus.cc:374] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:11.542220 30633 raft_consensus.cc:729] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d00bb9d0ca4949d7b0aae57fd6beaf02, State: Initialized, Role: FOLLOWER
I20260929 19:14:11.542351 30633 consensus_queue.cc:260] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00bb9d0ca4949d7b0aae57fd6beaf02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34383 } }
I20260929 19:14:11.542431 30633 raft_consensus.cc:388] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:11.542454 30633 raft_consensus.cc:482] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:11.542503 30633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:11.543092 30633 raft_consensus.cc:504] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00bb9d0ca4949d7b0aae57fd6beaf02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34383 } }
I20260929 19:14:11.543188 30633 leader_election.cc:302] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d00bb9d0ca4949d7b0aae57fd6beaf02; no voters:
I20260929 19:14:11.543355 30633 leader_election.cc:288] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:11.543427 30643 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:11.543618 30643 raft_consensus.cc:686] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02 [term 1 LEADER]: Becoming Leader. State: Replica: d00bb9d0ca4949d7b0aae57fd6beaf02, State: Running, Role: LEADER
I20260929 19:14:11.543632 30633 sys_catalog.cc:564] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:11.543756 30643 consensus_queue.cc:237] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00bb9d0ca4949d7b0aae57fd6beaf02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34383 } }
I20260929 19:14:11.544201 30645 sys_catalog.cc:455] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d00bb9d0ca4949d7b0aae57fd6beaf02. Latest consensus state: current_term: 1 leader_uuid: "d00bb9d0ca4949d7b0aae57fd6beaf02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00bb9d0ca4949d7b0aae57fd6beaf02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34383 } } }
I20260929 19:14:11.544211 30644 sys_catalog.cc:455] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d00bb9d0ca4949d7b0aae57fd6beaf02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00bb9d0ca4949d7b0aae57fd6beaf02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34383 } } }
I20260929 19:14:11.544404 30645 sys_catalog.cc:458] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:11.544427 30644 sys_catalog.cc:458] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:11.544759 30652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:11.545384 30652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:11.547081 30652 catalog_manager.cc:1347] Generated new cluster ID: 6d9c0b0728de40dd9c69a64f875f3d12
I20260929 19:14:11.547133 30652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:11.556362 30652 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:11.556506 30652 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:11.559973 30652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d00bb9d0ca4949d7b0aae57fd6beaf02: Generated new TSK 0
I20260929 19:14:11.560119 30652 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:11.568532 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 30565
2026-09-29T19:14:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1222 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:11 dist-test-slave-wftq krb5kdc[30664](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:11 dist-test-slave-wftq krb5kdc[30664](info): set up 2 sockets
Sep 29 19:14:11 dist-test-slave-wftq krb5kdc[30664](info): commencing operation
krb5kdc: starting...
W20260929 19:14:12.513325 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:12 dist-test-slave-wftq krb5kdc[30664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:14:12.637297 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41951
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:12.718091 30677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:12.718295 30677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:12.718317 30677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:12.719437 30677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:12.719483 30677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:12.719496 30677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:12.719508 30677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:12.719522 30677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:12.720959 30677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:12.721191 30677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:12.721395 30677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:12.726542 30683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:12.726575 30682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:12.726563 30685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:12.726593 30677 server_base.cc:1034] running on GCE node
I20260929 19:14:12.727089 30677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:12.727401 30677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:12.728569 30677 hybrid_clock.cc:648] HybridClock initialized: now 1790709252728550 us; error 42 us; skew 500 ppm
Sep 29 19:14:12 dist-test-slave-wftq krb5kdc[30664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709252, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:12.731166 30677 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:14:12.731601 30677 webserver.cc:460] Webserver started at http://127.0.0.1:43681/ using document root <none> and password file <none>
I20260929 19:14:12.731818 30677 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:12.731863 30677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:12.731992 30677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:12.733199 30677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "f69a687e0a15460d99bbb8af436e9d72"
format_stamp: "Formatted at 2026-09-29 19:14:12 on dist-test-slave-wftq"
server_key: "7b9a7cf1a8089f7e6bd8d02ceb3faf93"
server_key_iv: "735eb423ca1cac8229f22c32cb38c39e"
server_key_version: "encryptionkey@0"
I20260929 19:14:12.733546 30677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "f69a687e0a15460d99bbb8af436e9d72"
format_stamp: "Formatted at 2026-09-29 19:14:12 on dist-test-slave-wftq"
server_key: "7b9a7cf1a8089f7e6bd8d02ceb3faf93"
server_key_iv: "735eb423ca1cac8229f22c32cb38c39e"
server_key_version: "encryptionkey@0"
I20260929 19:14:12.734858 30677 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:14:12.735610 30692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:12.735809 30677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:14:12.735888 30677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "f69a687e0a15460d99bbb8af436e9d72"
format_stamp: "Formatted at 2026-09-29 19:14:12 on dist-test-slave-wftq"
server_key: "7b9a7cf1a8089f7e6bd8d02ceb3faf93"
server_key_iv: "735eb423ca1cac8229f22c32cb38c39e"
server_key_version: "encryptionkey@0"
I20260929 19:14:12.735991 30677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:12.754730 30677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:12.756700 30677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:12.756820 30677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:12.761248 30677 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41951
I20260929 19:14:12.761301 30744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41951 every 8 connection(s)
I20260929 19:14:12.761662 30677 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:12.762552 30745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:12.763377 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 30677
I20260929 19:14:12.763495 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:12.763696 10160 external_mini_cluster.cc:1428] Setting key 51b056db8222b55441f2fa06c11585b9
I20260929 19:14:12.765141 30745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72: Bootstrap starting.
I20260929 19:14:12.765733 30745 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:12.766014 30745 log.cc:824] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:12.766808 30745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72: No bootstrap required, opened a new log
I20260929 19:14:12.768968 30745 raft_consensus.cc:348] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69a687e0a15460d99bbb8af436e9d72" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41951 } }
I20260929 19:14:12.769156 30745 raft_consensus.cc:374] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:12.769202 30745 raft_consensus.cc:729] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f69a687e0a15460d99bbb8af436e9d72, State: Initialized, Role: FOLLOWER
I20260929 19:14:12.769351 30745 consensus_queue.cc:260] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69a687e0a15460d99bbb8af436e9d72" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41951 } }
I20260929 19:14:12.769429 30745 raft_consensus.cc:388] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:12.769461 30745 raft_consensus.cc:482] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:12.769502 30745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:12.770056 30745 raft_consensus.cc:504] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69a687e0a15460d99bbb8af436e9d72" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41951 } }
I20260929 19:14:12.770148 30745 leader_election.cc:302] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f69a687e0a15460d99bbb8af436e9d72; no voters:
I20260929 19:14:12.770313 30745 leader_election.cc:288] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:12.770354 30750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:12.770474 30750 raft_consensus.cc:686] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72 [term 1 LEADER]: Becoming Leader. State: Replica: f69a687e0a15460d99bbb8af436e9d72, State: Running, Role: LEADER
I20260929 19:14:12.770557 30750 consensus_queue.cc:237] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69a687e0a15460d99bbb8af436e9d72" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41951 } }
I20260929 19:14:12.770579 30745 sys_catalog.cc:564] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72 [sys.catalog]: configured and running, proceeding with master startup.
Sep 29 19:14:12 dist-test-slave-wftq krb5kdc[30664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 19:14:12.771847 30752 sys_catalog.cc:455] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f69a687e0a15460d99bbb8af436e9d72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69a687e0a15460d99bbb8af436e9d72" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41951 } } }
I20260929 19:14:12.771941 30752 sys_catalog.cc:458] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:12.772127 30751 sys_catalog.cc:455] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f69a687e0a15460d99bbb8af436e9d72. Latest consensus state: current_term: 1 leader_uuid: "f69a687e0a15460d99bbb8af436e9d72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69a687e0a15460d99bbb8af436e9d72" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41951 } } }
I20260929 19:14:12.772137 30766 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:12.772226 30751 sys_catalog.cc:458] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:12.772650 30766 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:12.775213 30748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:12.764257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54981 (local address 127.0.0.1:41951)
0929 19:14:12.764462 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:12.764471 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:12.764499 (+ 28us) server_negotiation.cc:407] Connection header received
0929 19:14:12.765045 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:12.765052 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:12.765143 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:12.765289 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:12.765976 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:12.767240 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:12.768070 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:12.768684 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:12.768822 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:12.771519 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:12.771526 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:12.771526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:12.771542 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:12.773409 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:12.773765 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:12.773768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:12.773769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:12.773877 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:12.774081 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:12.774082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:12.774083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:12.774452 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:12.774604 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:12.774839 (+ 235us) server_negotiation.cc:299] Negotiation successful
0929 19:14:12.774984 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":105,"threads_started":1}
I20260929 19:14:12.775365 30766 catalog_manager.cc:1347] Generated new cluster ID: 6dc15fccdbb642f1a59e9b9b3ab4d336
I20260929 19:14:12.775414 30766 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:12.776144 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:14:12.786530 30748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:12.777582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42401 (local address 127.0.0.1:41951)
0929 19:14:12.777616 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:12.777618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:12.781265 (+ 3647us) server_negotiation.cc:407] Connection header received
0929 19:14:12.781320 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:12.781321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:12.781331 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:12.781381 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:12.781851 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:12.782354 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:12.783039 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:12.783260 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:12.783313 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:12.784323 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:12.784325 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:12.784325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:12.784331 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:12.785571 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:12.785846 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:12.785847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:12.785847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:12.785915 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:12.786069 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:12.786070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:12.786070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:12.786175 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:12.786261 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:12.786365 (+ 104us) server_negotiation.cc:299] Negotiation successful
0929 19:14:12.786440 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":16}
I20260929 19:14:12.791958 30766 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:12.792701 30766 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:12.796993 30766 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f69a687e0a15460d99bbb8af436e9d72: Generated new TSK 0
I20260929 19:14:12.797197 30766 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:12.803299 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 30677
2026-09-29T19:14:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1234 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:12 dist-test-slave-wftq krb5kdc[30776](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:12 dist-test-slave-wftq krb5kdc[30776](info): set up 2 sockets
Sep 29 19:14:12 dist-test-slave-wftq krb5kdc[30776](info): commencing operation
krb5kdc: starting...
W20260929 19:14:13.761317 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:13 dist-test-slave-wftq krb5kdc[30776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:14:13.883677 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33797
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:13.961416 30789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:13.961621 30789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:13.961643 30789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:13.962720 30789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:13.962759 30789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:13.962771 30789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:13.962782 30789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:13.962793 30789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:13.964203 30789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:13.964388 30789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:13.964543 30789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:13.969542 30794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:13.969564 30795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:13.969558 30797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:13.969750 30789 server_base.cc:1034] running on GCE node
I20260929 19:14:13.970160 30789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:13.970533 30789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:13.971694 30789 hybrid_clock.cc:648] HybridClock initialized: now 1790709253971670 us; error 42 us; skew 500 ppm
Sep 29 19:14:13 dist-test-slave-wftq krb5kdc[30776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709253, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:13.974532 30789 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:14:13.975006 30789 webserver.cc:460] Webserver started at http://127.0.0.1:34503/ using document root <none> and password file <none>
I20260929 19:14:13.975227 30789 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:13.975270 30789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:13.975373 30789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:13.976528 30789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "2e269763c5b84dcda15e34c6512a691d"
format_stamp: "Formatted at 2026-09-29 19:14:13 on dist-test-slave-wftq"
server_key: "efb6b846111b97a783065c187b8b3efb"
server_key_iv: "eefd28723dfc7e4aa4f782b2c01d7374"
server_key_version: "encryptionkey@0"
I20260929 19:14:13.976850 30789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "2e269763c5b84dcda15e34c6512a691d"
format_stamp: "Formatted at 2026-09-29 19:14:13 on dist-test-slave-wftq"
server_key: "efb6b846111b97a783065c187b8b3efb"
server_key_iv: "eefd28723dfc7e4aa4f782b2c01d7374"
server_key_version: "encryptionkey@0"
I20260929 19:14:13.978307 30789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:14:13.979164 30804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:13.979391 30789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:14:13.979482 30789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "2e269763c5b84dcda15e34c6512a691d"
format_stamp: "Formatted at 2026-09-29 19:14:13 on dist-test-slave-wftq"
server_key: "efb6b846111b97a783065c187b8b3efb"
server_key_iv: "eefd28723dfc7e4aa4f782b2c01d7374"
server_key_version: "encryptionkey@0"
I20260929 19:14:13.979570 30789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:14.001157 30789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:14.003270 30789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:14.003386 30789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:14.007382 30789 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33797
I20260929 19:14:14.007486 30856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33797 every 8 connection(s)
I20260929 19:14:14.007784 30789 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:14.008879 30857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:14.009198 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 30789
I20260929 19:14:14.009352 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:14.009600 10160 external_mini_cluster.cc:1428] Setting key c59c926c3b31bd8da92c763251a114d1
I20260929 19:14:14.012980 30857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d: Bootstrap starting.
I20260929 19:14:14.013837 30857 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:14.014178 30857 log.cc:824] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:14.015097 30857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d: No bootstrap required, opened a new log
I20260929 19:14:14.015591 30860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:14.010226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33235 (local address 127.0.0.1:33797)
0929 19:14:14.010420 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:14.010429 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:14.010456 (+ 27us) server_negotiation.cc:407] Connection header received
0929 19:14:14.010998 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:14.011005 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:14.011052 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:14.011177 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:14.011497 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:14.012390 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:14.013565 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:14.013993 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:14.014085 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:14.014833 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:14.014837 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:14.014837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:14.015049 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:14.015086 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:14.015101 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:14.015176 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:14.015313 (+ 137us) server_negotiation.cc:299] Negotiation successful
0929 19:14:14.015381 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":89,"threads_started":1}
I20260929 19:14:14.016798 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:14:14.017390 30857 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e269763c5b84dcda15e34c6512a691d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33797 } }
I20260929 19:14:14.017580 30857 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:14.017616 30857 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e269763c5b84dcda15e34c6512a691d, State: Initialized, Role: FOLLOWER
I20260929 19:14:14.017750 30857 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e269763c5b84dcda15e34c6512a691d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33797 } }
I20260929 19:14:14.017835 30857 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:14.017877 30857 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:14.017935 30857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:14.018723 30857 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e269763c5b84dcda15e34c6512a691d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33797 } }
I20260929 19:14:14.018816 30857 leader_election.cc:302] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e269763c5b84dcda15e34c6512a691d; no voters:
I20260929 19:14:14.018971 30857 leader_election.cc:288] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:14.019059 30867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:14.019225 30857 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:14.019243 30867 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d [term 1 LEADER]: Becoming Leader. State: Replica: 2e269763c5b84dcda15e34c6512a691d, State: Running, Role: LEADER
I20260929 19:14:14.019418 30867 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e269763c5b84dcda15e34c6512a691d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33797 } }
I20260929 19:14:14.019852 30869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e269763c5b84dcda15e34c6512a691d. Latest consensus state: current_term: 1 leader_uuid: "2e269763c5b84dcda15e34c6512a691d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e269763c5b84dcda15e34c6512a691d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33797 } } }
I20260929 19:14:14.019913 30868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e269763c5b84dcda15e34c6512a691d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e269763c5b84dcda15e34c6512a691d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33797 } } }
I20260929 19:14:14.019984 30869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:14.020004 30868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d [sys.catalog]: This master's current role is: LEADER
W20260929 19:14:14.021291 30882 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:14:14.021353 30882 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:14:14.021400 30876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:14.021575 30876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:14.023243 30876 catalog_manager.cc:1347] Generated new cluster ID: 4ea4609f6009419881167e5ec6d8e00f
I20260929 19:14:14.023293 30876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:14.024142 30860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:14.018449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57391 (local address 127.0.0.1:33797)
0929 19:14:14.018508 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:14.018510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:14.020779 (+ 2269us) server_negotiation.cc:407] Connection header received
0929 19:14:14.020914 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:14.020915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:14.020930 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:14.021001 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:14.021249 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:14.021941 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:14.022775 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:14.023036 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:14.023105 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:14.023890 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:14.023891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:14.023892 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:14.023897 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:14.023914 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:14.023918 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:14.023971 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:14.024049 (+ 78us) server_negotiation.cc:299] Negotiation successful
0929 19:14:14.024071 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 19:14:14.052444 30876 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:14.052973 30876 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:14.057626 30876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e269763c5b84dcda15e34c6512a691d: Generated new TSK 0
I20260929 19:14:14.057914 30876 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:14.064663 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 30789
2026-09-29T19:14:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1262 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:14 dist-test-slave-wftq krb5kdc[30888](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:14 dist-test-slave-wftq krb5kdc[30888](info): set up 2 sockets
Sep 29 19:14:14 dist-test-slave-wftq krb5kdc[30888](info): commencing operation
krb5kdc: starting...
W20260929 19:14:15.013345 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:15 dist-test-slave-wftq krb5kdc[30888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:14:15.135332 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42701
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:15.222644 30901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:15.222884 30901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:15.222908 30901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:15.224064 30901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:15.224104 30901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:15.224118 30901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:15.224128 30901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:15.224139 30901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:15.225600 30901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:15.225808 30901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:15.226015 30901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:15.231127 30907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:15.231079 30906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:15.231418 30901 server_base.cc:1034] running on GCE node
W20260929 19:14:15.231192 30909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:15.231695 30901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:15.232007 30901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:15.233152 30901 hybrid_clock.cc:648] HybridClock initialized: now 1790709255233142 us; error 29 us; skew 500 ppm
Sep 29 19:14:15 dist-test-slave-wftq krb5kdc[30888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709255, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:15.235716 30901 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:14:15.236225 30901 webserver.cc:460] Webserver started at http://127.0.0.1:42053/ using document root <none> and password file <none>
I20260929 19:14:15.236439 30901 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:15.236487 30901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:15.236614 30901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:15.237782 30901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "f75dec4834f448fea6dcef57afdc46c1"
format_stamp: "Formatted at 2026-09-29 19:14:15 on dist-test-slave-wftq"
server_key: "d26265a9dc155e44b737412802c54c14"
server_key_iv: "a854f61cf7c2dc11bad740643f45f1d5"
server_key_version: "encryptionkey@0"
I20260929 19:14:15.238133 30901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "f75dec4834f448fea6dcef57afdc46c1"
format_stamp: "Formatted at 2026-09-29 19:14:15 on dist-test-slave-wftq"
server_key: "d26265a9dc155e44b737412802c54c14"
server_key_iv: "a854f61cf7c2dc11bad740643f45f1d5"
server_key_version: "encryptionkey@0"
I20260929 19:14:15.239531 30901 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:14:15.240396 30916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:15.240574 30901 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:15.240693 30901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "f75dec4834f448fea6dcef57afdc46c1"
format_stamp: "Formatted at 2026-09-29 19:14:15 on dist-test-slave-wftq"
server_key: "d26265a9dc155e44b737412802c54c14"
server_key_iv: "a854f61cf7c2dc11bad740643f45f1d5"
server_key_version: "encryptionkey@0"
I20260929 19:14:15.240794 30901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:15.244155 30901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:15.246203 30901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:15.246315 30901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:15.250349 30901 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42701
I20260929 19:14:15.250411 30968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42701 every 8 connection(s)
I20260929 19:14:15.250713 30901 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:15.251754 30969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:15.254336 30969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1: Bootstrap starting.
I20260929 19:14:15.254904 30969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:15.255177 30969 log.cc:824] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:15.255928 30969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1: No bootstrap required, opened a new log
I20260929 19:14:15.257651 30969 raft_consensus.cc:348] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f75dec4834f448fea6dcef57afdc46c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42701 } }
I20260929 19:14:15.257797 30969 raft_consensus.cc:374] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:15.257819 30969 raft_consensus.cc:729] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75dec4834f448fea6dcef57afdc46c1, State: Initialized, Role: FOLLOWER
I20260929 19:14:15.257946 30969 consensus_queue.cc:260] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f75dec4834f448fea6dcef57afdc46c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42701 } }
I20260929 19:14:15.258028 30969 raft_consensus.cc:388] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:15.258062 30969 raft_consensus.cc:482] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:15.258098 30969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:15.258596 30969 raft_consensus.cc:504] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f75dec4834f448fea6dcef57afdc46c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42701 } }
I20260929 19:14:15.258684 30969 leader_election.cc:302] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f75dec4834f448fea6dcef57afdc46c1; no voters:
I20260929 19:14:15.258816 30969 leader_election.cc:288] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:15.258882 30972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:15.259079 30972 raft_consensus.cc:686] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1 [term 1 LEADER]: Becoming Leader. State: Replica: f75dec4834f448fea6dcef57afdc46c1, State: Running, Role: LEADER
I20260929 19:14:15.259086 30969 sys_catalog.cc:564] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:15.259274 30972 consensus_queue.cc:237] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f75dec4834f448fea6dcef57afdc46c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42701 } }
I20260929 19:14:15.259908 30974 sys_catalog.cc:455] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f75dec4834f448fea6dcef57afdc46c1. Latest consensus state: current_term: 1 leader_uuid: "f75dec4834f448fea6dcef57afdc46c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f75dec4834f448fea6dcef57afdc46c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42701 } } }
I20260929 19:14:15.260003 30974 sys_catalog.cc:458] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:15.260147 30973 sys_catalog.cc:455] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f75dec4834f448fea6dcef57afdc46c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f75dec4834f448fea6dcef57afdc46c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42701 } } }
I20260929 19:14:15.260243 30973 sys_catalog.cc:458] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:15.260550 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 30901
I20260929 19:14:15.260680 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:15.260864 10160 external_mini_cluster.cc:1428] Setting key f8484f83f63f746e9d1d6b0228ef663e
I20260929 19:14:15.260708 30981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:15.261608 30981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:15.263384 30981 catalog_manager.cc:1347] Generated new cluster ID: 3d9e7b5625654a4da361f8d64a6fc939
I20260929 19:14:15.263441 30981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:15.263561 30991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:15.261396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46245 (local address 127.0.0.1:42701)
0929 19:14:15.261726 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:15.261736 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:15.261768 (+ 32us) server_negotiation.cc:407] Connection header received
0929 19:14:15.261907 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:15.261915 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:15.261947 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:15.262387 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:15.262952 (+ 565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:15.262956 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:15.262956 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:15.262960 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:15.263000 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:15.263015 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:15.263275 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:15.263307 (+ 32us) server_negotiation.cc:299] Negotiation successful
0929 19:14:15.263352 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":110,"threads_started":1}
I20260929 19:14:15.264438 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:14:15.272163 30991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:15.266064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36571 (local address 127.0.0.1:42701)
0929 19:14:15.266101 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:15.266102 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:15.266111 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:14:15.266151 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:15.266152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:15.266158 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:15.266197 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:15.271941 (+ 5744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:15.271942 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:15.271943 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:15.271947 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:15.271967 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:15.271971 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:15.272019 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:15.272072 (+ 53us) server_negotiation.cc:299] Negotiation successful
0929 19:14:15.272086 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:14:15.303138 30981 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:15.303335 30981 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:15.309484 30981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f75dec4834f448fea6dcef57afdc46c1: Generated new TSK 0
I20260929 19:14:15.309674 30981 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:15.317888 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 30901
2026-09-29T19:14:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1253 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:15 dist-test-slave-wftq krb5kdc[31000](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:15 dist-test-slave-wftq krb5kdc[31000](info): set up 2 sockets
Sep 29 19:14:15 dist-test-slave-wftq krb5kdc[31000](info): commencing operation
krb5kdc: starting...
W20260929 19:14:16.277485 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:16 dist-test-slave-wftq krb5kdc[31000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:14:16.397615 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44957
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:16.483022 31013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:16.483269 31013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:16.483300 31013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:16.484783 31013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:16.484853 31013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:16.484879 31013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:16.484899 31013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:16.484918 31013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:16.486572 31013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:16.486876 31013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:16.487106 31013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:16.492390 31019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:16.492403 31021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:16.492403 31018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:16.492563 31013 server_base.cc:1034] running on GCE node
I20260929 19:14:16.492945 31013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:16.493294 31013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:16.494431 31013 hybrid_clock.cc:648] HybridClock initialized: now 1790709256494421 us; error 46 us; skew 500 ppm
Sep 29 19:14:16 dist-test-slave-wftq krb5kdc[31000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709256, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:16.497068 31013 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:14:16.497577 31013 webserver.cc:460] Webserver started at http://127.0.0.1:33199/ using document root <none> and password file <none>
I20260929 19:14:16.497805 31013 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:16.497853 31013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:16.497983 31013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:16.499059 31013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "cc433c1af8d34e0a8d1198d4371076c7"
format_stamp: "Formatted at 2026-09-29 19:14:16 on dist-test-slave-wftq"
server_key: "016e0e410a6af27a9562e25cc2e1b71e"
server_key_iv: "e70e3a566873d6e579c80b028be605a2"
server_key_version: "encryptionkey@0"
I20260929 19:14:16.499405 31013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "cc433c1af8d34e0a8d1198d4371076c7"
format_stamp: "Formatted at 2026-09-29 19:14:16 on dist-test-slave-wftq"
server_key: "016e0e410a6af27a9562e25cc2e1b71e"
server_key_iv: "e70e3a566873d6e579c80b028be605a2"
server_key_version: "encryptionkey@0"
I20260929 19:14:16.500762 31013 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:14:16.501524 31028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:16.501711 31013 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 19:14:16.501796 31013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "cc433c1af8d34e0a8d1198d4371076c7"
format_stamp: "Formatted at 2026-09-29 19:14:16 on dist-test-slave-wftq"
server_key: "016e0e410a6af27a9562e25cc2e1b71e"
server_key_iv: "e70e3a566873d6e579c80b028be605a2"
server_key_version: "encryptionkey@0"
I20260929 19:14:16.501855 31013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:16.525043 31013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:16.526918 31013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:16.527048 31013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:16.531260 31013 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44957
I20260929 19:14:16.531330 31080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44957 every 8 connection(s)
I20260929 19:14:16.531683 31013 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:16.532658 31081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:16.532857 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 31013
I20260929 19:14:16.532998 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:16.533259 10160 external_mini_cluster.cc:1428] Setting key 2b44246b2040d850bf48c876e8cb9d34
I20260929 19:14:16.535610 31081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7: Bootstrap starting.
I20260929 19:14:16.536276 31081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:16.536546 31081 log.cc:824] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:16.537230 31081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7: No bootstrap required, opened a new log
I20260929 19:14:16.539356 31081 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc433c1af8d34e0a8d1198d4371076c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44957 } }
I20260929 19:14:16.539544 31081 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:16.539578 31081 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc433c1af8d34e0a8d1198d4371076c7, State: Initialized, Role: FOLLOWER
I20260929 19:14:16.539723 31081 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc433c1af8d34e0a8d1198d4371076c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44957 } }
I20260929 19:14:16.539803 31081 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:16.539841 31081 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:16.539901 31081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7 [term 0 FOLLOWER]: Advancing to term 1
Sep 29 19:14:16 dist-test-slave-wftq krb5kdc[31000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 19:14:16.540462 31081 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc433c1af8d34e0a8d1198d4371076c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44957 } }
I20260929 19:14:16.540551 31081 leader_election.cc:302] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc433c1af8d34e0a8d1198d4371076c7; no voters:
I20260929 19:14:16.540716 31081 leader_election.cc:288] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:16.540759 31086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:16.540877 31086 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7 [term 1 LEADER]: Becoming Leader. State: Replica: cc433c1af8d34e0a8d1198d4371076c7, State: Running, Role: LEADER
I20260929 19:14:16.540959 31086 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc433c1af8d34e0a8d1198d4371076c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44957 } }
I20260929 19:14:16.540989 31081 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:16.541440 31087 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc433c1af8d34e0a8d1198d4371076c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc433c1af8d34e0a8d1198d4371076c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44957 } } }
I20260929 19:14:16.541522 31087 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:16.541692 31088 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc433c1af8d34e0a8d1198d4371076c7. Latest consensus state: current_term: 1 leader_uuid: "cc433c1af8d34e0a8d1198d4371076c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc433c1af8d34e0a8d1198d4371076c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44957 } } }
I20260929 19:14:16.541756 31091 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:16.542375 31091 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:16.543030 31088 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:16.544179 31091 catalog_manager.cc:1347] Generated new cluster ID: 526eceb9b34b4788bd5ec9e8615fb718
I20260929 19:14:16.544277 31091 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:16.545658 31084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:16.533872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:42459 (local address 127.0.0.1:44957)
0929 19:14:16.534084 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:16.534093 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:16.534119 (+ 26us) server_negotiation.cc:407] Connection header received
0929 19:14:16.534700 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:16.534707 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:16.534760 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:16.534887 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:16.535483 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:16.536517 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:16.537390 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:16.537835 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:16.537958 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:16.540367 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:16.540371 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:16.540371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:16.540383 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:16.543573 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:16.544160 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:16.544164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:16.544165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:16.544347 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:16.544631 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:16.544631 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:16.544632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:16.544979 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:16.545190 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:16.545294 (+ 104us) server_negotiation.cc:299] Negotiation successful
0929 19:14:16.545415 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":102,"threads_started":1}
I20260929 19:14:16.546773 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:14:16.555814 31084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:16.548517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:44893 (local address 127.0.0.1:44957)
0929 19:14:16.548557 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:16.548558 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:16.548623 (+ 65us) server_negotiation.cc:407] Connection header received
0929 19:14:16.548695 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:16.548696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:16.548707 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:16.548762 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:16.549368 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:16.550180 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:16.551149 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:16.551478 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:16.551556 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:16.552735 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:16.552736 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:16.552737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:16.552744 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:16.554567 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:16.555011 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:16.555012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:16.555013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:16.555078 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:16.555292 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:16.555293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:16.555293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:16.555435 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:16.555541 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:16.555651 (+ 110us) server_negotiation.cc:299] Negotiation successful
0929 19:14:16.555716 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:14:16.558113 31091 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:16.558610 31091 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:16.567209 31091 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc433c1af8d34e0a8d1198d4371076c7: Generated new TSK 0
I20260929 19:14:16.567453 31091 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:16.573961 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 31013
W20260929 19:14:16.578155 30998 connection.cc:535] client connection to 127.0.0.1:44957 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44957): Connection reset by peer (error 104)
2026-09-29T19:14:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1254 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:16 dist-test-slave-wftq krb5kdc[31112](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:16 dist-test-slave-wftq krb5kdc[31112](info): set up 2 sockets
Sep 29 19:14:16 dist-test-slave-wftq krb5kdc[31112](info): commencing operation
krb5kdc: starting...
W20260929 19:14:17.517488 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:17 dist-test-slave-wftq krb5kdc[31112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:14:17.640024 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38167
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:17.725042 31125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:17.725274 31125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:17.725306 31125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:17.726424 31125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:17.726468 31125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:17.726487 31125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:17.726504 31125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:17.726519 31125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:17.727988 31125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:17.728229 31125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:17.728432 31125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:17.733541 31130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:17.733613 31133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:17.733541 31131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:17.733603 31125 server_base.cc:1034] running on GCE node
I20260929 19:14:17.734081 31125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:17.734342 31125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:17.735478 31125 hybrid_clock.cc:648] HybridClock initialized: now 1790709257735466 us; error 37 us; skew 500 ppm
Sep 29 19:14:17 dist-test-slave-wftq krb5kdc[31112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709257, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:17.737952 31125 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:14:17.738396 31125 webserver.cc:460] Webserver started at http://127.0.0.1:33255/ using document root <none> and password file <none>
I20260929 19:14:17.738601 31125 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:17.738646 31125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:17.738797 31125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:17.739805 31125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "3e9385fb5cb04f50bea1991f0eb9a950"
format_stamp: "Formatted at 2026-09-29 19:14:17 on dist-test-slave-wftq"
server_key: "0a4c9efa96b0c09218184120f5139e5e"
server_key_iv: "1542e1c417daf2c2ccf84b0cffc91894"
server_key_version: "encryptionkey@0"
I20260929 19:14:17.740110 31125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "3e9385fb5cb04f50bea1991f0eb9a950"
format_stamp: "Formatted at 2026-09-29 19:14:17 on dist-test-slave-wftq"
server_key: "0a4c9efa96b0c09218184120f5139e5e"
server_key_iv: "1542e1c417daf2c2ccf84b0cffc91894"
server_key_version: "encryptionkey@0"
I20260929 19:14:17.741365 31125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 19:14:17.742079 31140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:17.742246 31125 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:14:17.742326 31125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "3e9385fb5cb04f50bea1991f0eb9a950"
format_stamp: "Formatted at 2026-09-29 19:14:17 on dist-test-slave-wftq"
server_key: "0a4c9efa96b0c09218184120f5139e5e"
server_key_iv: "1542e1c417daf2c2ccf84b0cffc91894"
server_key_version: "encryptionkey@0"
I20260929 19:14:17.742394 31125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:17.754541 31125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:17.756373 31125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:17.756489 31125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:17.760599 31125 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38167
I20260929 19:14:17.760677 31192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38167 every 8 connection(s)
I20260929 19:14:17.760952 31125 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:17.762019 31193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:17.764572 31193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950: Bootstrap starting.
I20260929 19:14:17.765264 31193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:17.765290 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 31125
I20260929 19:14:17.765401 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:17.765607 31193 log.cc:824] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:17.765646 10160 external_mini_cluster.cc:1428] Setting key 2066b4d0bc9aeab832326b0adf39b474
I20260929 19:14:17.766484 31193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950: No bootstrap required, opened a new log
I20260929 19:14:17.768944 31193 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e9385fb5cb04f50bea1991f0eb9a950" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38167 } }
I20260929 19:14:17.769133 31193 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:17.769182 31193 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9385fb5cb04f50bea1991f0eb9a950, State: Initialized, Role: FOLLOWER
I20260929 19:14:17.769318 31193 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e9385fb5cb04f50bea1991f0eb9a950" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38167 } }
I20260929 19:14:17.769408 31193 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:17.769440 31193 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:17.769488 31193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:17.770103 31193 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e9385fb5cb04f50bea1991f0eb9a950" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38167 } }
I20260929 19:14:17.770202 31193 leader_election.cc:302] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e9385fb5cb04f50bea1991f0eb9a950; no voters:
I20260929 19:14:17.770373 31193 leader_election.cc:288] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:17.770433 31198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:17.770649 31193 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:17.770676 31198 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950 [term 1 LEADER]: Becoming Leader. State: Replica: 3e9385fb5cb04f50bea1991f0eb9a950, State: Running, Role: LEADER
I20260929 19:14:17.770838 31198 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e9385fb5cb04f50bea1991f0eb9a950" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38167 } }
I20260929 19:14:17.771297 31199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e9385fb5cb04f50bea1991f0eb9a950" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e9385fb5cb04f50bea1991f0eb9a950" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38167 } } }
I20260929 19:14:17.771438 31199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:17.771478 31197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:17.766164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:43805 (local address 127.0.0.1:38167)
0929 19:14:17.766366 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:17.766377 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:17.766413 (+ 36us) server_negotiation.cc:407] Connection header received
0929 19:14:17.767048 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:17.767055 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:17.767108 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:17.767242 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:17.767429 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:17.768341 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:17.769078 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:17.769714 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:17.769811 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:17.770535 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:17.770541 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:17.770541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:17.770779 (+ 238us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:17.770827 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:17.770856 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:17.770985 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:17.771106 (+ 121us) server_negotiation.cc:299] Negotiation successful
0929 19:14:17.771142 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":84,"threads_started":1}
I20260929 19:14:17.771725 31207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:17.771255 31200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e9385fb5cb04f50bea1991f0eb9a950. Latest consensus state: current_term: 1 leader_uuid: "3e9385fb5cb04f50bea1991f0eb9a950" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e9385fb5cb04f50bea1991f0eb9a950" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38167 } } }
I20260929 19:14:17.772396 31200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:17.772496 31207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:17.772540 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:14:17.774667 31207 catalog_manager.cc:1347] Generated new cluster ID: fa149d94a95142b297ac6f7881522137
I20260929 19:14:17.774717 31207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:17.777819 31197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:17.774053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:38823 (local address 127.0.0.1:38167)
0929 19:14:17.774093 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:17.774094 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:17.774132 (+ 38us) server_negotiation.cc:407] Connection header received
0929 19:14:17.774222 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:17.774223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:17.774236 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:17.774296 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:17.774453 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:17.775218 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:17.776337 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:17.776647 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:17.776689 (+ 42us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:17.777551 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:17.777553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:17.777553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:17.777557 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:17.777572 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:17.777577 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:17.777609 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:17.777735 (+ 126us) server_negotiation.cc:299] Negotiation successful
0929 19:14:17.777747 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:14:17.784019 31207 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:17.784531 31207 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:17.788797 31207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e9385fb5cb04f50bea1991f0eb9a950: Generated new TSK 0
I20260929 19:14:17.788937 31207 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:17.797025 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 31125
W20260929 19:14:17.801647 31110 connection.cc:535] client connection to 127.0.0.1:38167 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38167): Connection reset by peer (error 104)
2026-09-29T19:14:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1223 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:17 dist-test-slave-wftq krb5kdc[31224](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:17 dist-test-slave-wftq krb5kdc[31224](info): set up 2 sockets
Sep 29 19:14:17 dist-test-slave-wftq krb5kdc[31224](info): commencing operation
krb5kdc: starting...
W20260929 19:14:18.749317 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:18 dist-test-slave-wftq krb5kdc[31224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:14:18.870363 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39945
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:18.948276 31237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:18.948490 31237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:18.948513 31237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:18.949632 31237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:18.949672 31237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:18.949685 31237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:18.949697 31237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:18.949707 31237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:18.951107 31237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:18.951301 31237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:18.951460 31237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:18.956187 31245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:18.956207 31243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:18.956225 31242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:18.956444 31237 server_base.cc:1034] running on GCE node
I20260929 19:14:18.956714 31237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:18.956974 31237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:18.958134 31237 hybrid_clock.cc:648] HybridClock initialized: now 1790709258958117 us; error 33 us; skew 500 ppm
Sep 29 19:14:18 dist-test-slave-wftq krb5kdc[31224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709258, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:18.960721 31237 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:14:18.961195 31237 webserver.cc:460] Webserver started at http://127.0.0.1:37877/ using document root <none> and password file <none>
I20260929 19:14:18.961408 31237 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:18.961457 31237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:18.961581 31237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:18.962620 31237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "c17f7823c8f54d62b2bcfd4880c26862"
format_stamp: "Formatted at 2026-09-29 19:14:18 on dist-test-slave-wftq"
server_key: "cfca80f9124f1f64a65e2540195b4eb4"
server_key_iv: "6c1ac565af67918c0faf7f10eec2bbe1"
server_key_version: "encryptionkey@0"
I20260929 19:14:18.962931 31237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "c17f7823c8f54d62b2bcfd4880c26862"
format_stamp: "Formatted at 2026-09-29 19:14:18 on dist-test-slave-wftq"
server_key: "cfca80f9124f1f64a65e2540195b4eb4"
server_key_iv: "6c1ac565af67918c0faf7f10eec2bbe1"
server_key_version: "encryptionkey@0"
I20260929 19:14:18.964179 31237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:14:18.964936 31252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:18.965154 31237 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:18.965255 31237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "c17f7823c8f54d62b2bcfd4880c26862"
format_stamp: "Formatted at 2026-09-29 19:14:18 on dist-test-slave-wftq"
server_key: "cfca80f9124f1f64a65e2540195b4eb4"
server_key_iv: "6c1ac565af67918c0faf7f10eec2bbe1"
server_key_version: "encryptionkey@0"
I20260929 19:14:18.965313 31237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:18.968333 31237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:18.970414 31237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:18.970526 31237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:18.974682 31237 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39945
I20260929 19:14:18.974759 31304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39945 every 8 connection(s)
I20260929 19:14:18.975041 31237 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:18.975443 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 31237
I20260929 19:14:18.975538 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:18.975715 10160 external_mini_cluster.cc:1428] Setting key e5e0aad33865354e8c740f6a3371649e
I20260929 19:14:18.976383 31305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:18.978156 31308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:18.976220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:34843 (local address 127.0.0.1:39945)
0929 19:14:18.976467 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:18.976477 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:18.976512 (+ 35us) server_negotiation.cc:407] Connection header received
0929 19:14:18.976971 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:18.976977 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:18.977001 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:18.977150 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:18.977779 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:18.977783 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:18.977783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:18.977787 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:18.977831 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:18.977842 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:18.977898 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:18.977954 (+ 56us) server_negotiation.cc:299] Negotiation successful
0929 19:14:18.977996 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":140,"threads_started":1}
I20260929 19:14:18.979110 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:14:18.979733 31305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862: Bootstrap starting.
I20260929 19:14:18.980420 31305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:18.980769 31305 log.cc:824] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:18.981446 31308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:18.980424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:55105 (local address 127.0.0.1:39945)
0929 19:14:18.980485 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:18.980487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:18.980495 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:14:18.980554 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:18.980554 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:18.980559 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:18.980601 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:18.981185 (+ 584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:18.981187 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:18.981187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:18.981192 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:18.981214 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:18.981219 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:18.981273 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:18.981365 (+ 92us) server_negotiation.cc:299] Negotiation successful
0929 19:14:18.981380 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 19:14:18.981722 31305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862: No bootstrap required, opened a new log
I20260929 19:14:18.983666 31305 raft_consensus.cc:348] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17f7823c8f54d62b2bcfd4880c26862" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39945 } }
I20260929 19:14:18.983834 31305 raft_consensus.cc:374] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:18.983857 31305 raft_consensus.cc:729] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c17f7823c8f54d62b2bcfd4880c26862, State: Initialized, Role: FOLLOWER
I20260929 19:14:18.983982 31305 consensus_queue.cc:260] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17f7823c8f54d62b2bcfd4880c26862" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39945 } }
I20260929 19:14:18.984069 31305 raft_consensus.cc:388] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:18.984094 31305 raft_consensus.cc:482] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:18.984145 31305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:18.984788 31305 raft_consensus.cc:504] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17f7823c8f54d62b2bcfd4880c26862" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39945 } }
I20260929 19:14:18.984897 31305 leader_election.cc:302] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c17f7823c8f54d62b2bcfd4880c26862; no voters:
I20260929 19:14:18.985080 31305 leader_election.cc:288] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:18.985145 31315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:18.985282 31315 raft_consensus.cc:686] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862 [term 1 LEADER]: Becoming Leader. State: Replica: c17f7823c8f54d62b2bcfd4880c26862, State: Running, Role: LEADER
I20260929 19:14:18.985410 31305 sys_catalog.cc:564] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:18.985446 31315 consensus_queue.cc:237] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17f7823c8f54d62b2bcfd4880c26862" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39945 } }
I20260929 19:14:18.985900 31316 sys_catalog.cc:455] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c17f7823c8f54d62b2bcfd4880c26862" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17f7823c8f54d62b2bcfd4880c26862" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39945 } } }
I20260929 19:14:18.985915 31317 sys_catalog.cc:455] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c17f7823c8f54d62b2bcfd4880c26862. Latest consensus state: current_term: 1 leader_uuid: "c17f7823c8f54d62b2bcfd4880c26862" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17f7823c8f54d62b2bcfd4880c26862" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39945 } } }
I20260929 19:14:18.986148 31316 sys_catalog.cc:458] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:18.986166 31317 sys_catalog.cc:458] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:18.986766 31324 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:18.987293 31324 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:18.988795 31324 catalog_manager.cc:1347] Generated new cluster ID: 2abca96828e64c45af7da90ff055e26e
I20260929 19:14:18.988852 31324 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:19.007493 31324 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:19.007778 31324 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:19.014247 31324 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c17f7823c8f54d62b2bcfd4880c26862: Generated new TSK 0
I20260929 19:14:19.014572 31324 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:19.023680 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 31237
2026-09-29T19:14:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1225 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:19 dist-test-slave-wftq krb5kdc[31336](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:19 dist-test-slave-wftq krb5kdc[31336](info): set up 2 sockets
Sep 29 19:14:19 dist-test-slave-wftq krb5kdc[31336](info): commencing operation
krb5kdc: starting...
W20260929 19:14:19.973295 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:20 dist-test-slave-wftq krb5kdc[31336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:14:20.093940 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45057
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:20.180250 31349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:20.180496 31349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:20.180519 31349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:20.181734 31349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:20.181774 31349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:20.181787 31349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:20.181799 31349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:20.181809 31349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:20.183317 31349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45057
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:20.183542 31349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:20.183768 31349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:20.188934 31357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:20.188993 31354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:20.189154 31349 server_base.cc:1034] running on GCE node
W20260929 19:14:20.189211 31355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:20.189487 31349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:20.189801 31349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:20.190950 31349 hybrid_clock.cc:648] HybridClock initialized: now 1790709260190941 us; error 28 us; skew 500 ppm
Sep 29 19:14:20 dist-test-slave-wftq krb5kdc[31336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709260, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:20.193516 31349 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:14:20.194001 31349 webserver.cc:460] Webserver started at http://127.0.0.1:44213/ using document root <none> and password file <none>
I20260929 19:14:20.194216 31349 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:20.194262 31349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:20.194389 31349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:20.195549 31349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "9ea3cb21a6a04cbbb2e9db658d6380fd"
format_stamp: "Formatted at 2026-09-29 19:14:20 on dist-test-slave-wftq"
server_key: "4c25294d747c4698fed98772fcc2c8c2"
server_key_iv: "65d9567ac50543677828f0977c8e9551"
server_key_version: "encryptionkey@0"
I20260929 19:14:20.195899 31349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "9ea3cb21a6a04cbbb2e9db658d6380fd"
format_stamp: "Formatted at 2026-09-29 19:14:20 on dist-test-slave-wftq"
server_key: "4c25294d747c4698fed98772fcc2c8c2"
server_key_iv: "65d9567ac50543677828f0977c8e9551"
server_key_version: "encryptionkey@0"
I20260929 19:14:20.197365 31349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:14:20.198261 31364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:20.198467 31349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:20.198555 31349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "9ea3cb21a6a04cbbb2e9db658d6380fd"
format_stamp: "Formatted at 2026-09-29 19:14:20 on dist-test-slave-wftq"
server_key: "4c25294d747c4698fed98772fcc2c8c2"
server_key_iv: "65d9567ac50543677828f0977c8e9551"
server_key_version: "encryptionkey@0"
I20260929 19:14:20.198643 31349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:20.218569 31349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:20.220572 31349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:20.220696 31349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:20.225431 31349 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45057
I20260929 19:14:20.225514 31416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45057 every 8 connection(s)
I20260929 19:14:20.225853 31349 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:20.226814 31417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:20.229199 31417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd: Bootstrap starting.
I20260929 19:14:20.229198 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 31349
I20260929 19:14:20.229290 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:20.229524 10160 external_mini_cluster.cc:1428] Setting key 660f03675e566cb2d4f3ad58d6e8e2e8
I20260929 19:14:20.229784 31417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:20.230074 31417 log.cc:824] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:20.230741 31417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd: No bootstrap required, opened a new log
I20260929 19:14:20.232730 31417 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea3cb21a6a04cbbb2e9db658d6380fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45057 } }
I20260929 19:14:20.232872 31417 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:20.232903 31417 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ea3cb21a6a04cbbb2e9db658d6380fd, State: Initialized, Role: FOLLOWER
I20260929 19:14:20.233035 31417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea3cb21a6a04cbbb2e9db658d6380fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45057 } }
I20260929 19:14:20.233134 31417 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:20.233175 31417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:20.233225 31417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:20.233774 31417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea3cb21a6a04cbbb2e9db658d6380fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45057 } }
I20260929 19:14:20.233872 31417 leader_election.cc:302] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ea3cb21a6a04cbbb2e9db658d6380fd; no voters:
I20260929 19:14:20.234057 31417 leader_election.cc:288] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:20.234097 31422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:20.234206 31422 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd [term 1 LEADER]: Becoming Leader. State: Replica: 9ea3cb21a6a04cbbb2e9db658d6380fd, State: Running, Role: LEADER
I20260929 19:14:20.234336 31417 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:20.234361 31422 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea3cb21a6a04cbbb2e9db658d6380fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45057 } }
I20260929 19:14:20.234773 31423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ea3cb21a6a04cbbb2e9db658d6380fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea3cb21a6a04cbbb2e9db658d6380fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45057 } } }
I20260929 19:14:20.234851 31423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:20.235080 31424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ea3cb21a6a04cbbb2e9db658d6380fd. Latest consensus state: current_term: 1 leader_uuid: "9ea3cb21a6a04cbbb2e9db658d6380fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea3cb21a6a04cbbb2e9db658d6380fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45057 } } }
I20260929 19:14:20.235208 31424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:20.235666 31426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:20.236274 31426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 29 19:14:20 dist-test-slave-wftq krb5kdc[31336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 19:14:20.238826 31426 catalog_manager.cc:1347] Generated new cluster ID: 52812059b687447b980ad2c6cfa0fad0
I20260929 19:14:20.238883 31426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:20.240806 31420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:20.230073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:43377 (local address 127.0.0.1:45057)
0929 19:14:20.230266 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:20.230278 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:20.230344 (+ 66us) server_negotiation.cc:407] Connection header received
0929 19:14:20.231018 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:20.231026 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:20.231089 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:20.231229 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:20.231864 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:20.232823 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:20.233713 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:20.234244 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:20.234715 (+ 471us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:20.237408 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:20.237413 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:20.237413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:20.237426 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:20.239242 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:20.239723 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:20.239726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:20.239726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:20.239813 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:20.240008 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:20.240008 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:20.240009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:20.240263 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:20.240431 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:20.240517 (+ 86us) server_negotiation.cc:299] Negotiation successful
0929 19:14:20.240636 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":66,"threads_started":1}
I20260929 19:14:20.242121 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:14:20.251106 31420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:20.245429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:41771 (local address 127.0.0.1:45057)
0929 19:14:20.245469 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:20.245471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:20.245647 (+ 176us) server_negotiation.cc:407] Connection header received
0929 19:14:20.245701 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:20.245701 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:20.245713 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:20.245756 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:20.246259 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:20.246777 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:20.247447 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:20.247683 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:20.247736 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:20.248823 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:20.248825 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:20.248825 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:20.248832 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:20.250094 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:20.250411 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:20.250412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:20.250412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:20.250461 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:20.250647 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:20.250648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:20.250648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:20.250751 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:20.250855 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:20.250948 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 19:14:20.251007 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:14:20.263162 31426 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:20.263789 31426 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:20.267511 31426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ea3cb21a6a04cbbb2e9db658d6380fd: Generated new TSK 0
I20260929 19:14:20.267731 31426 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:20.285590 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 31349
W20260929 19:14:20.290321 31334 connection.cc:535] client connection to 127.0.0.1:45057 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45057): Connection reset by peer (error 104)
2026-09-29T19:14:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1263 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:20 dist-test-slave-wftq krb5kdc[31448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:20 dist-test-slave-wftq krb5kdc[31448](info): set up 2 sockets
Sep 29 19:14:20 dist-test-slave-wftq krb5kdc[31448](info): commencing operation
krb5kdc: starting...
W20260929 19:14:21.253278 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:21 dist-test-slave-wftq krb5kdc[31448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:14:21.376287 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46445
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:21.457891 31461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:21.458093 31461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:21.458120 31461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:21.459220 31461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:21.459263 31461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:21.459281 31461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:21.459298 31461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:21.459312 31461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:21.460747 31461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46445
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:21.460970 31461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:21.461186 31461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:14:21.466295 31461 server_base.cc:1034] running on GCE node
W20260929 19:14:21.466248 31469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:21.466284 31467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:21.466264 31466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:21.466682 31461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:21.466943 31461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:21.468081 31461 hybrid_clock.cc:648] HybridClock initialized: now 1790709261468059 us; error 44 us; skew 500 ppm
Sep 29 19:14:21 dist-test-slave-wftq krb5kdc[31448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709261, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:21.470422 31461 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:14:21.470832 31461 webserver.cc:460] Webserver started at http://127.0.0.1:43157/ using document root <none> and password file <none>
I20260929 19:14:21.471002 31461 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:21.471042 31461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:21.471138 31461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:21.472105 31461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "f379f274d5b8474c9f1930df4b761742"
format_stamp: "Formatted at 2026-09-29 19:14:21 on dist-test-slave-wftq"
server_key: "62070897e53c8582e94ab189fbf7f2e8"
server_key_iv: "acc981c41f3d8c1f927c0ae586666d46"
server_key_version: "encryptionkey@0"
I20260929 19:14:21.472388 31461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "f379f274d5b8474c9f1930df4b761742"
format_stamp: "Formatted at 2026-09-29 19:14:21 on dist-test-slave-wftq"
server_key: "62070897e53c8582e94ab189fbf7f2e8"
server_key_iv: "acc981c41f3d8c1f927c0ae586666d46"
server_key_version: "encryptionkey@0"
I20260929 19:14:21.473688 31461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260929 19:14:21.474439 31476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:21.474629 31461 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:21.474713 31461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "f379f274d5b8474c9f1930df4b761742"
format_stamp: "Formatted at 2026-09-29 19:14:21 on dist-test-slave-wftq"
server_key: "62070897e53c8582e94ab189fbf7f2e8"
server_key_iv: "acc981c41f3d8c1f927c0ae586666d46"
server_key_version: "encryptionkey@0"
I20260929 19:14:21.474785 31461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:21.496817 31461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:21.498843 31461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:21.498972 31461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:21.503113 31461 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46445
I20260929 19:14:21.503178 31528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46445 every 8 connection(s)
I20260929 19:14:21.503481 31461 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:21.504499 31529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:21.507174 31529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742: Bootstrap starting.
I20260929 19:14:21.507797 31529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:21.508072 31529 log.cc:824] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:21.508816 31529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742: No bootstrap required, opened a new log
I20260929 19:14:21.510535 31529 raft_consensus.cc:348] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f379f274d5b8474c9f1930df4b761742" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46445 } }
I20260929 19:14:21.510679 31529 raft_consensus.cc:374] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:21.510712 31529 raft_consensus.cc:729] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f379f274d5b8474c9f1930df4b761742, State: Initialized, Role: FOLLOWER
I20260929 19:14:21.510859 31529 consensus_queue.cc:260] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f379f274d5b8474c9f1930df4b761742" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46445 } }
I20260929 19:14:21.510923 31529 raft_consensus.cc:388] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:21.510953 31529 raft_consensus.cc:482] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:21.511000 31529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:21.511638 31529 raft_consensus.cc:504] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f379f274d5b8474c9f1930df4b761742" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46445 } }
I20260929 19:14:21.511736 31529 leader_election.cc:302] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f379f274d5b8474c9f1930df4b761742; no voters:
I20260929 19:14:21.511900 31529 leader_election.cc:288] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:21.511950 31532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:21.511940 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 31461
I20260929 19:14:21.512051 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:21.512085 31532 raft_consensus.cc:686] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742 [term 1 LEADER]: Becoming Leader. State: Replica: f379f274d5b8474c9f1930df4b761742, State: Running, Role: LEADER
I20260929 19:14:21.512157 31532 consensus_queue.cc:237] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f379f274d5b8474c9f1930df4b761742" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46445 } }
I20260929 19:14:21.512187 31529 sys_catalog.cc:564] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:21.512269 10160 external_mini_cluster.cc:1428] Setting key 482d22bdcf16afa8c3609ba3d1ddd8c2
I20260929 19:14:21.512737 31534 sys_catalog.cc:455] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f379f274d5b8474c9f1930df4b761742. Latest consensus state: current_term: 1 leader_uuid: "f379f274d5b8474c9f1930df4b761742" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f379f274d5b8474c9f1930df4b761742" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46445 } } }
I20260929 19:14:21.512882 31534 sys_catalog.cc:458] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:21.513139 31543 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:21.513283 31533 sys_catalog.cc:455] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f379f274d5b8474c9f1930df4b761742" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f379f274d5b8474c9f1930df4b761742" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46445 } } }
I20260929 19:14:21.513378 31533 sys_catalog.cc:458] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:21.513783 31543 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:21.515578 31543 catalog_manager.cc:1347] Generated new cluster ID: 9efd2e394dc642a3ba77c73a548113ef
I20260929 19:14:21.515633 31543 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:21.518128 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:21.512790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:34721 (local address 127.0.0.1:46445)
0929 19:14:21.513247 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:21.513261 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:21.513307 (+ 46us) server_negotiation.cc:407] Connection header received
0929 19:14:21.513523 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:21.513532 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:21.513600 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:21.513741 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:21.514271 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:21.515632 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:21.516379 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:21.516793 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:21.516887 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:21.517530 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:21.517533 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:21.517534 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:21.517710 (+ 176us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:21.517741 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:21.517752 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:21.517820 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:21.517915 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 19:14:21.517943 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":97,"threads_started":1}
I20260929 19:14:21.519127 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:14:21.523053 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:21.520381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:34191 (local address 127.0.0.1:46445)
0929 19:14:21.520417 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:21.520420 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:21.520482 (+ 62us) server_negotiation.cc:407] Connection header received
0929 19:14:21.520537 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:21.520538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:21.520562 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:21.520612 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:21.520735 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:21.521286 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:21.521959 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:21.522200 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:21.522253 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:21.522831 (+ 578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:21.522832 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:21.522833 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:21.522837 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:21.522852 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:21.522857 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:21.522900 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:21.522965 (+ 65us) server_negotiation.cc:299] Negotiation successful
0929 19:14:21.522978 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 19:14:21.534161 31543 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:21.534749 31543 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:21.539047 31543 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f379f274d5b8474c9f1930df4b761742: Generated new TSK 0
I20260929 19:14:21.539206 31543 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:21.545292 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 31461
W20260929 19:14:21.550559 31446 connection.cc:535] client connection to 127.0.0.1:46445 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46445): Connection reset by peer (error 104)
2026-09-29T19:14:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1259 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:21 dist-test-slave-wftq krb5kdc[31560](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:21 dist-test-slave-wftq krb5kdc[31560](info): set up 2 sockets
Sep 29 19:14:21 dist-test-slave-wftq krb5kdc[31560](info): commencing operation
krb5kdc: starting...
W20260929 19:14:22.493057 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:22 dist-test-slave-wftq krb5kdc[31560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 19:14:22.613425 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38907
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:22.695715 31573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:22.695922 31573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:22.695945 31573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:22.697085 31573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:22.697166 31573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:22.697182 31573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:22.697196 31573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:22.697224 31573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:22.698750 31573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38907
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:22.698973 31573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:22.699180 31573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:22.704577 31579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:22.704617 31573 server_base.cc:1034] running on GCE node
W20260929 19:14:22.704536 31581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:22.704564 31578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:22.705238 31573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:22.705703 31573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:22.706861 31573 hybrid_clock.cc:648] HybridClock initialized: now 1790709262706837 us; error 37 us; skew 500 ppm
Sep 29 19:14:22 dist-test-slave-wftq krb5kdc[31560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709262, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:22.709510 31573 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 19:14:22.709990 31573 webserver.cc:460] Webserver started at http://127.0.0.1:43181/ using document root <none> and password file <none>
I20260929 19:14:22.710227 31573 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:22.710273 31573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:22.710412 31573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:22.711561 31573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "b46f4f3f39934a9c9318c25232f939c8"
format_stamp: "Formatted at 2026-09-29 19:14:22 on dist-test-slave-wftq"
server_key: "f72860e1f7345567c876c6fb1b8e5e41"
server_key_iv: "efacc19c9b0b0b61626144724aa909df"
server_key_version: "encryptionkey@0"
I20260929 19:14:22.711905 31573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "b46f4f3f39934a9c9318c25232f939c8"
format_stamp: "Formatted at 2026-09-29 19:14:22 on dist-test-slave-wftq"
server_key: "f72860e1f7345567c876c6fb1b8e5e41"
server_key_iv: "efacc19c9b0b0b61626144724aa909df"
server_key_version: "encryptionkey@0"
I20260929 19:14:22.713348 31573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:14:22.714195 31588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:22.714424 31573 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:22.714504 31573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "b46f4f3f39934a9c9318c25232f939c8"
format_stamp: "Formatted at 2026-09-29 19:14:22 on dist-test-slave-wftq"
server_key: "f72860e1f7345567c876c6fb1b8e5e41"
server_key_iv: "efacc19c9b0b0b61626144724aa909df"
server_key_version: "encryptionkey@0"
I20260929 19:14:22.714569 31573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:22.718041 31573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:22.720024 31573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:22.720153 31573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:22.724526 31573 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38907
I20260929 19:14:22.724587 31640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38907 every 8 connection(s)
I20260929 19:14:22.724891 31573 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:22.725971 31641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:22.728220 31641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8: Bootstrap starting.
I20260929 19:14:22.728453 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 31573
I20260929 19:14:22.728562 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:22.728752 10160 external_mini_cluster.cc:1428] Setting key dd024acbdd1e7f4de25cecd131a4746b
I20260929 19:14:22.728904 31641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:22.729250 31641 log.cc:824] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:22.730053 31641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8: No bootstrap required, opened a new log
I20260929 19:14:22.731340 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:22.729348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:39063 (local address 127.0.0.1:38907)
0929 19:14:22.729616 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:22.729641 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:22.729682 (+ 41us) server_negotiation.cc:407] Connection header received
0929 19:14:22.730134 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:22.730143 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:22.730177 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:22.730319 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:22.730972 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:22.730975 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:22.730976 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:22.730979 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:22.731014 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:22.731024 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:22.731095 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:22.731159 (+ 64us) server_negotiation.cc:299] Negotiation successful
0929 19:14:22.731193 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":168,"threads_started":1}
I20260929 19:14:22.732513 31641 raft_consensus.cc:348] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46f4f3f39934a9c9318c25232f939c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38907 } }
I20260929 19:14:22.732673 31641 raft_consensus.cc:374] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:22.732695 31641 raft_consensus.cc:729] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b46f4f3f39934a9c9318c25232f939c8, State: Initialized, Role: FOLLOWER
I20260929 19:14:22.732815 31641 consensus_queue.cc:260] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46f4f3f39934a9c9318c25232f939c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38907 } }
I20260929 19:14:22.732908 31641 raft_consensus.cc:388] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:22.732941 31641 raft_consensus.cc:482] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:22.732983 31641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:22.733791 31641 raft_consensus.cc:504] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46f4f3f39934a9c9318c25232f939c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38907 } }
I20260929 19:14:22.733894 31641 leader_election.cc:302] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b46f4f3f39934a9c9318c25232f939c8; no voters:
I20260929 19:14:22.734074 31641 leader_election.cc:288] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:22.734086 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:14:22.734104 31646 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:22.734234 31646 raft_consensus.cc:686] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8 [term 1 LEADER]: Becoming Leader. State: Replica: b46f4f3f39934a9c9318c25232f939c8, State: Running, Role: LEADER
I20260929 19:14:22.734359 31641 sys_catalog.cc:564] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:22.734360 31646 consensus_queue.cc:237] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46f4f3f39934a9c9318c25232f939c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38907 } }
I20260929 19:14:22.734894 31647 sys_catalog.cc:455] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b46f4f3f39934a9c9318c25232f939c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46f4f3f39934a9c9318c25232f939c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38907 } } }
I20260929 19:14:22.734882 31648 sys_catalog.cc:455] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b46f4f3f39934a9c9318c25232f939c8. Latest consensus state: current_term: 1 leader_uuid: "b46f4f3f39934a9c9318c25232f939c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46f4f3f39934a9c9318c25232f939c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38907 } } }
I20260929 19:14:22.735029 31647 sys_catalog.cc:458] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:22.735044 31648 sys_catalog.cc:458] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:22.735532 31656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:22.736136 31656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:22.738364 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:22.737181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:40765 (local address 127.0.0.1:38907)
0929 19:14:22.737231 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:22.737233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:22.737304 (+ 71us) server_negotiation.cc:407] Connection header received
0929 19:14:22.737362 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:22.737363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:22.737370 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:22.737421 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:22.738089 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:22.738090 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:22.738091 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:22.738095 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:22.738113 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:22.738119 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:22.738170 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:22.738286 (+ 116us) server_negotiation.cc:299] Negotiation successful
0929 19:14:22.738301 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 19:14:22.738693 31656 catalog_manager.cc:1347] Generated new cluster ID: a8815bd5dc044196bd9fd78e1aaadec3
I20260929 19:14:22.738742 31656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:22.752153 31656 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:22.752331 31656 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:22.757129 31656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b46f4f3f39934a9c9318c25232f939c8: Generated new TSK 0
I20260929 19:14:22.757323 31656 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:22.762970 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 31573
2026-09-29T19:14:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1217 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:22 dist-test-slave-wftq krb5kdc[31672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:22 dist-test-slave-wftq krb5kdc[31672](info): set up 2 sockets
Sep 29 19:14:22 dist-test-slave-wftq krb5kdc[31672](info): commencing operation
krb5kdc: starting...
W20260929 19:14:23.713572 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:23 dist-test-slave-wftq krb5kdc[31672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:14:23.835426 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:42011
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:42011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:23.913705 31685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:23.913918 31685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:23.913947 31685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:23.915071 31685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:23.915114 31685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:23.915132 31685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:23.915148 31685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:23.915162 31685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:23.916643 31685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:42011
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:42011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:23.916873 31685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:23.917070 31685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:23.921864 31693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:23.921975 31691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:23.921977 31690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:23.922169 31685 server_base.cc:1034] running on GCE node
I20260929 19:14:23.922480 31685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:23.922835 31685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:23.923990 31685 hybrid_clock.cc:648] HybridClock initialized: now 1790709263923962 us; error 46 us; skew 500 ppm
Sep 29 19:14:23 dist-test-slave-wftq krb5kdc[31672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709263, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:23.926772 31685 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:14:23.927253 31685 webserver.cc:460] Webserver started at http://127.9.236.62:45903/ using document root <none> and password file <none>
I20260929 19:14:23.927479 31685 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:23.927532 31685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:23.927667 31685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:23.928856 31685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "be555161108e42cd997857a7007eb4ba"
format_stamp: "Formatted at 2026-09-29 19:14:23 on dist-test-slave-wftq"
server_key: "220daa672213e48d3ad125cef799f9ca"
server_key_iv: "9e96c355cfc339b68012b3463c24af70"
server_key_version: "encryptionkey@0"
I20260929 19:14:23.929253 31685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "be555161108e42cd997857a7007eb4ba"
format_stamp: "Formatted at 2026-09-29 19:14:23 on dist-test-slave-wftq"
server_key: "220daa672213e48d3ad125cef799f9ca"
server_key_iv: "9e96c355cfc339b68012b3463c24af70"
server_key_version: "encryptionkey@0"
I20260929 19:14:23.930728 31685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:14:23.931537 31700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:23.931774 31685 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:14:23.931874 31685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "be555161108e42cd997857a7007eb4ba"
format_stamp: "Formatted at 2026-09-29 19:14:23 on dist-test-slave-wftq"
server_key: "220daa672213e48d3ad125cef799f9ca"
server_key_iv: "9e96c355cfc339b68012b3463c24af70"
server_key_version: "encryptionkey@0"
I20260929 19:14:23.931942 31685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:23.955456 31685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:23.957448 31685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:23.957602 31685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:23.962495 31685 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:42011
I20260929 19:14:23.962563 31752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:42011 every 8 connection(s)
I20260929 19:14:23.962987 31685 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:23.963935 31753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:23.966245 31753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba: Bootstrap starting.
I20260929 19:14:23.966764 31753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:23.966985 31753 log.cc:824] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:23.967711 31753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba: No bootstrap required, opened a new log
I20260929 19:14:23.969287 31753 raft_consensus.cc:348] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be555161108e42cd997857a7007eb4ba" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 42011 } }
I20260929 19:14:23.969425 31753 raft_consensus.cc:374] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:23.969445 31753 raft_consensus.cc:729] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be555161108e42cd997857a7007eb4ba, State: Initialized, Role: FOLLOWER
I20260929 19:14:23.969555 31753 consensus_queue.cc:260] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be555161108e42cd997857a7007eb4ba" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 42011 } }
I20260929 19:14:23.969628 31753 raft_consensus.cc:388] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:23.969662 31753 raft_consensus.cc:482] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:23.969700 31753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:23.970471 31753 raft_consensus.cc:504] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be555161108e42cd997857a7007eb4ba" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 42011 } }
I20260929 19:14:23.970557 31753 leader_election.cc:302] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be555161108e42cd997857a7007eb4ba; no voters:
I20260929 19:14:23.970688 31753 leader_election.cc:288] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:23.970851 31757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:23.970894 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 31685
I20260929 19:14:23.970924 31753 sys_catalog.cc:564] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:23.971014 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:23.971127 31757 raft_consensus.cc:686] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba [term 1 LEADER]: Becoming Leader. State: Replica: be555161108e42cd997857a7007eb4ba, State: Running, Role: LEADER
I20260929 19:14:23.971251 10160 external_mini_cluster.cc:1428] Setting key 0827804d0839cea710fb0fe4ddb3d3e0
I20260929 19:14:23.971293 31757 consensus_queue.cc:237] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be555161108e42cd997857a7007eb4ba" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 42011 } }
I20260929 19:14:23.971815 31756 sys_catalog.cc:455] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be555161108e42cd997857a7007eb4ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be555161108e42cd997857a7007eb4ba" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 42011 } } }
I20260929 19:14:23.971832 31758 sys_catalog.cc:455] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader be555161108e42cd997857a7007eb4ba. Latest consensus state: current_term: 1 leader_uuid: "be555161108e42cd997857a7007eb4ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be555161108e42cd997857a7007eb4ba" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 42011 } } }
I20260929 19:14:23.971937 31756 sys_catalog.cc:458] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:23.971966 31758 sys_catalog.cc:458] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba [sys.catalog]: This master's current role is: LEADER
W20260929 19:14:23.973294 31773 catalog_manager.cc:1558] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:14:23.973418 31773 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:14:23.973495 31767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:23.973719 31767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:23.975762 31767 catalog_manager.cc:1347] Generated new cluster ID: a3a37623c178475ba4c6d4b5bce5cc2b
I20260929 19:14:23.975824 31767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:14:23 dist-test-slave-wftq krb5kdc[31672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:14:23.981310 31766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:23.971809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41167 (local address 127.9.236.62:42011)
0929 19:14:23.972107 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:23.972117 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:23.972226 (+ 109us) server_negotiation.cc:407] Connection header received
0929 19:14:23.972361 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:23.972369 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:23.972430 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:23.973188 (+ 758us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:23.973397 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:23.974562 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:23.975721 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:23.976160 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:23.976262 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:23.978444 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:23.978448 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:23.978449 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:23.978459 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:23.979924 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:23.980267 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:23.980269 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:23.980270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:23.980358 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:23.980548 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:23.980549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:23.980549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:23.980778 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:23.980943 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:23.981025 (+ 82us) server_negotiation.cc:299] Negotiation successful
0929 19:14:23.981160 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":75,"threads_started":1}
I20260929 19:14:23.982198 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:14:23.994961 31766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:23.985411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53457 (local address 127.9.236.62:42011)
0929 19:14:23.985446 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:23.985448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:23.989260 (+ 3812us) server_negotiation.cc:407] Connection header received
0929 19:14:23.989310 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:23.989311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:23.989321 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:23.989368 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:23.989871 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:23.990383 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:23.991039 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:23.991262 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:23.991315 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:23.992606 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:23.992607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:23.992608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:23.992614 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:23.993895 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:23.994207 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:23.994208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:23.994209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:23.994262 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:23.994441 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:23.994441 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:23.994442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:23.994544 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:23.994632 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:23.994740 (+ 108us) server_negotiation.cc:299] Negotiation successful
0929 19:14:23.994800 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 19:14:24.013588 31767 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:24.014394 31767 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:24.028554 31767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be555161108e42cd997857a7007eb4ba: Generated new TSK 0
I20260929 19:14:24.028771 31767 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:24.037338 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 31685
W20260929 19:14:24.041290 31670 connection.cc:535] client connection to 127.9.236.62:42011 recv error: Network error: failed to read from TLS socket (remote: 127.9.236.62:42011): Connection reset by peer (error 104)
2026-09-29T19:14:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1273 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:24 dist-test-slave-wftq krb5kdc[31784](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:24 dist-test-slave-wftq krb5kdc[31784](info): set up 2 sockets
Sep 29 19:14:24 dist-test-slave-wftq krb5kdc[31784](info): commencing operation
krb5kdc: starting...
W20260929 19:14:24.985303 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:25 dist-test-slave-wftq krb5kdc[31784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:14:25.101926 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:36681
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:43109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:36681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:25.181699 31797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:25.181972 31797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:25.182009 31797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:25.183382 31797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:25.183424 31797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:25.183439 31797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:25.183452 31797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:25.183465 31797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:25.185200 31797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:43109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:36681
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:36681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:25.185453 31797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:25.185679 31797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:25.191035 31803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:25.191071 31797 server_base.cc:1034] running on GCE node
W20260929 19:14:25.191021 31805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:25.191030 31802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:25.191550 31797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:25.191876 31797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:25.193042 31797 hybrid_clock.cc:648] HybridClock initialized: now 1790709265193031 us; error 30 us; skew 500 ppm
Sep 29 19:14:25 dist-test-slave-wftq krb5kdc[31784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709265, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:25.195788 31797 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:14:25.196331 31797 webserver.cc:460] Webserver started at http://127.9.236.62:44361/ using document root <none> and password file <none>
I20260929 19:14:25.196580 31797 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:25.196635 31797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:25.196767 31797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:25.198022 31797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "4c6e49f48fbf4cdcb42657f755c1852a"
format_stamp: "Formatted at 2026-09-29 19:14:25 on dist-test-slave-wftq"
server_key: "2ba5896e62ab1029cf4f93cf63e508e1"
server_key_iv: "c681e3f9d7e02ef7fb167cfc7b4fd470"
server_key_version: "encryptionkey@0"
I20260929 19:14:25.198366 31797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "4c6e49f48fbf4cdcb42657f755c1852a"
format_stamp: "Formatted at 2026-09-29 19:14:25 on dist-test-slave-wftq"
server_key: "2ba5896e62ab1029cf4f93cf63e508e1"
server_key_iv: "c681e3f9d7e02ef7fb167cfc7b4fd470"
server_key_version: "encryptionkey@0"
I20260929 19:14:25.199753 31797 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260929 19:14:25.200650 31812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:25.200981 31797 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:25.201082 31797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "4c6e49f48fbf4cdcb42657f755c1852a"
format_stamp: "Formatted at 2026-09-29 19:14:25 on dist-test-slave-wftq"
server_key: "2ba5896e62ab1029cf4f93cf63e508e1"
server_key_iv: "c681e3f9d7e02ef7fb167cfc7b4fd470"
server_key_version: "encryptionkey@0"
I20260929 19:14:25.201184 31797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:25.218554 31797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:25.220707 31797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:25.220860 31797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:25.225432 31797 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:36681
I20260929 19:14:25.225487 31864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:36681 every 8 connection(s)
I20260929 19:14:25.225857 31797 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:25.226811 31865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:25.227135 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 31797
I20260929 19:14:25.227232 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:25.227452 10160 external_mini_cluster.cc:1428] Setting key 018fa34448813a03e565b9e549cf22cb
I20260929 19:14:25.229880 31865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a: Bootstrap starting.
I20260929 19:14:25.230535 31865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:25.230822 31865 log.cc:824] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:25.231578 31865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a: No bootstrap required, opened a new log
I20260929 19:14:25.233788 31868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:25.228006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59385 (local address 127.9.236.62:36681)
0929 19:14:25.228238 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:25.228248 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:25.228284 (+ 36us) server_negotiation.cc:407] Connection header received
0929 19:14:25.228775 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:25.228783 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:25.228841 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:25.229002 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:25.229250 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:25.230548 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:25.231567 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:25.232180 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:25.232292 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:25.233049 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:25.233055 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:25.233055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:25.233293 (+ 238us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:25.233338 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:25.233353 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:25.233447 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:25.233550 (+ 103us) server_negotiation.cc:299] Negotiation successful
0929 19:14:25.233581 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":95,"threads_started":1}
I20260929 19:14:25.233902 31865 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c6e49f48fbf4cdcb42657f755c1852a" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36681 } }
I20260929 19:14:25.234043 31865 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:25.234074 31865 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c6e49f48fbf4cdcb42657f755c1852a, State: Initialized, Role: FOLLOWER
I20260929 19:14:25.234207 31865 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c6e49f48fbf4cdcb42657f755c1852a" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36681 } }
I20260929 19:14:25.234289 31865 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:25.234315 31865 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:25.234361 31865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:25.234977 31865 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c6e49f48fbf4cdcb42657f755c1852a" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36681 } }
I20260929 19:14:25.235026 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:14:25.235085 31865 leader_election.cc:302] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c6e49f48fbf4cdcb42657f755c1852a; no voters:
I20260929 19:14:25.235239 31865 leader_election.cc:288] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:25.235320 31870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:25.235474 31865 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:25.235555 31870 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a [term 1 LEADER]: Becoming Leader. State: Replica: 4c6e49f48fbf4cdcb42657f755c1852a, State: Running, Role: LEADER
I20260929 19:14:25.235709 31870 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c6e49f48fbf4cdcb42657f755c1852a" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36681 } }
I20260929 19:14:25.236430 31871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c6e49f48fbf4cdcb42657f755c1852a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c6e49f48fbf4cdcb42657f755c1852a" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36681 } } }
I20260929 19:14:25.236582 31871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:25.236768 31872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c6e49f48fbf4cdcb42657f755c1852a. Latest consensus state: current_term: 1 leader_uuid: "4c6e49f48fbf4cdcb42657f755c1852a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c6e49f48fbf4cdcb42657f755c1852a" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36681 } } }
I20260929 19:14:25.236829 31872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:25.236857 31883 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:25.237426 31883 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:25.239233 31883 catalog_manager.cc:1347] Generated new cluster ID: b39265b21cb1452b8018ed5cd8866b69
I20260929 19:14:25.239291 31883 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:25.241410 31868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:25.237223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38567 (local address 127.9.236.62:36681)
0929 19:14:25.237268 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:25.237270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:25.238324 (+ 1054us) server_negotiation.cc:407] Connection header received
0929 19:14:25.238382 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:25.238382 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:25.238396 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:25.238463 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:25.238679 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:25.239314 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:25.240088 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:25.240333 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:25.240396 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:25.241083 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:25.241085 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:25.241085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:25.241090 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:25.241144 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:25.241150 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:25.241206 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:25.241318 (+ 112us) server_negotiation.cc:299] Negotiation successful
0929 19:14:25.241328 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:14:25.255999 31883 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:25.256772 31883 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:25.263104 31883 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c6e49f48fbf4cdcb42657f755c1852a: Generated new TSK 0
I20260929 19:14:25.263283 31883 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:25.267153 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 31797
W20260929 19:14:25.271332 31782 connection.cc:535] client connection to 127.9.236.62:36681 recv error: Network error: failed to read from TLS socket (remote: 127.9.236.62:36681): Connection reset by peer (error 104)
2026-09-29T19:14:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1229 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:25 dist-test-slave-wftq krb5kdc[31896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:25 dist-test-slave-wftq krb5kdc[31896](info): set up 2 sockets
Sep 29 19:14:25 dist-test-slave-wftq krb5kdc[31896](info): commencing operation
krb5kdc: starting...
W20260929 19:14:26.229550 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.945s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:26 dist-test-slave-wftq krb5kdc[31896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:14:26.350392 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:43695
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:38113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:43695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:26.436455 31909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:26.436681 31909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:26.436703 31909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:26.437908 31909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:26.437954 31909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:26.437968 31909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:26.437978 31909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:26.437990 31909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:26.439477 31909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:38113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:43695
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:43695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:26.439731 31909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:26.439966 31909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:26.445189 31914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:26.445180 31917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:26.445379 31909 server_base.cc:1034] running on GCE node
W20260929 19:14:26.445178 31915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:26.445710 31909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:26.446033 31909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:26.447188 31909 hybrid_clock.cc:648] HybridClock initialized: now 1790709266447172 us; error 33 us; skew 500 ppm
Sep 29 19:14:26 dist-test-slave-wftq krb5kdc[31896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709266, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:26.450727 31909 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:14:26.451270 31909 webserver.cc:460] Webserver started at http://127.9.236.62:38439/ using document root <none> and password file <none>
I20260929 19:14:26.451555 31909 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:26.451612 31909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:26.451772 31909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:26.453207 31909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "9b6c307fec254e59a3b768154bef78c7"
format_stamp: "Formatted at 2026-09-29 19:14:26 on dist-test-slave-wftq"
server_key: "f3cfbcd6a929251ccbc780c4df1006c3"
server_key_iv: "5c5239e68b5b5f838add8c0a96eefb3d"
server_key_version: "encryptionkey@0"
I20260929 19:14:26.453661 31909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "9b6c307fec254e59a3b768154bef78c7"
format_stamp: "Formatted at 2026-09-29 19:14:26 on dist-test-slave-wftq"
server_key: "f3cfbcd6a929251ccbc780c4df1006c3"
server_key_iv: "5c5239e68b5b5f838add8c0a96eefb3d"
server_key_version: "encryptionkey@0"
I20260929 19:14:26.455214 31909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:14:26.456033 31924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:26.456311 31909 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:14:26.456415 31909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "9b6c307fec254e59a3b768154bef78c7"
format_stamp: "Formatted at 2026-09-29 19:14:26 on dist-test-slave-wftq"
server_key: "f3cfbcd6a929251ccbc780c4df1006c3"
server_key_iv: "5c5239e68b5b5f838add8c0a96eefb3d"
server_key_version: "encryptionkey@0"
I20260929 19:14:26.456493 31909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:26.461412 31909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:26.463961 31909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:26.464159 31909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:26.469825 31909 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:43695
I20260929 19:14:26.469918 31976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:43695 every 8 connection(s)
I20260929 19:14:26.470230 31909 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:26.471563 31977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:26.475332 31977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7: Bootstrap starting.
I20260929 19:14:26.475422 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 31909
I20260929 19:14:26.475530 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:26.475745 10160 external_mini_cluster.cc:1428] Setting key d9e596fc83030f36e1edaaeef53a2ce9
I20260929 19:14:26.476114 31977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:26.476467 31977 log.cc:824] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:26.477326 31977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7: No bootstrap required, opened a new log
I20260929 19:14:26.479461 31977 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b6c307fec254e59a3b768154bef78c7" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43695 } }
I20260929 19:14:26.479707 31977 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:26.479787 31977 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b6c307fec254e59a3b768154bef78c7, State: Initialized, Role: FOLLOWER
I20260929 19:14:26.479982 31977 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b6c307fec254e59a3b768154bef78c7" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43695 } }
I20260929 19:14:26.480129 31977 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:26.480192 31977 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:26.480265 31977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:26.481467 31980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:26.476487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60189 (local address 127.9.236.62:43695)
0929 19:14:26.478067 (+ 1580us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:26.478080 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:26.478126 (+ 46us) server_negotiation.cc:407] Connection header received
0929 19:14:26.479425 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:26.479437 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:26.479482 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:26.479646 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:26.480734 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:26.480740 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:26.480740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:26.480745 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:26.480815 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:26.480832 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:26.480888 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:26.481194 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 19:14:26.481247 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1490,"thread_start_us":86,"threads_started":1}
I20260929 19:14:26.481539 31977 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b6c307fec254e59a3b768154bef78c7" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43695 } }
I20260929 19:14:26.482551 31977 leader_election.cc:302] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b6c307fec254e59a3b768154bef78c7; no voters:
I20260929 19:14:26.482755 31977 leader_election.cc:288] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:26.483023 31983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:26.483067 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:14:26.483085 31977 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:26.483225 31983 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7 [term 1 LEADER]: Becoming Leader. State: Replica: 9b6c307fec254e59a3b768154bef78c7, State: Running, Role: LEADER
I20260929 19:14:26.483461 31983 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b6c307fec254e59a3b768154bef78c7" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43695 } }
I20260929 19:14:26.483903 31983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b6c307fec254e59a3b768154bef78c7. Latest consensus state: current_term: 1 leader_uuid: "9b6c307fec254e59a3b768154bef78c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b6c307fec254e59a3b768154bef78c7" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43695 } } }
I20260929 19:14:26.483994 31983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:26.485147 31984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b6c307fec254e59a3b768154bef78c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b6c307fec254e59a3b768154bef78c7" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 43695 } } }
I20260929 19:14:26.485517 31984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7 [sys.catalog]: This master's current role is: LEADER
W20260929 19:14:26.485649 32001 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:14:26.486099 32001 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:14:26.486202 32002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:26.486387 32002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:26.487464 31980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:26.484964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38879 (local address 127.9.236.62:43695)
0929 19:14:26.485004 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:26.485006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:26.485706 (+ 700us) server_negotiation.cc:407] Connection header received
0929 19:14:26.485785 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:26.485786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:26.485803 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:26.485850 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:26.487238 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:26.487240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:26.487240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:26.487244 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:26.487260 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:26.487264 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:26.487292 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:26.487390 (+ 98us) server_negotiation.cc:299] Negotiation successful
0929 19:14:26.487405 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260929 19:14:26.491722 32002 catalog_manager.cc:1347] Generated new cluster ID: 9e55517036624ad4a6f2b9793e5782d7
I20260929 19:14:26.491788 32002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:26.512562 32002 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:26.512735 32002 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:26.517025 32002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b6c307fec254e59a3b768154bef78c7: Generated new TSK 0
I20260929 19:14:26.517223 32002 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:26.526860 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 31909
2026-09-29T19:14:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1259 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:26 dist-test-slave-wftq krb5kdc[32008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:26 dist-test-slave-wftq krb5kdc[32008](info): set up 2 sockets
Sep 29 19:14:26 dist-test-slave-wftq krb5kdc[32008](info): commencing operation
krb5kdc: starting...
W20260929 19:14:27.481305 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:27 dist-test-slave-wftq krb5kdc[32008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:14:27.603766 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:38659
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:36483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:38659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:27.693665 32021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:27.693915 32021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:27.693939 32021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:27.695122 32021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:27.695171 32021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:27.695186 32021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:27.695199 32021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:27.695211 32021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:27.696749 32021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:36483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:38659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:38659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:27.696982 32021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:27.697248 32021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:27.702473 32029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:27.702430 32027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:27.702699 32021 server_base.cc:1034] running on GCE node
W20260929 19:14:27.702454 32026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:27.703115 32021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:27.703433 32021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:27.704588 32021 hybrid_clock.cc:648] HybridClock initialized: now 1790709267704599 us; error 61 us; skew 500 ppm
Sep 29 19:14:27 dist-test-slave-wftq krb5kdc[32008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709267, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:27.707275 32021 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:14:27.707770 32021 webserver.cc:460] Webserver started at http://127.9.236.62:33627/ using document root <none> and password file <none>
I20260929 19:14:27.708019 32021 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:27.708068 32021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:27.708194 32021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:27.709404 32021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "9af55692ae2a4d5eba662c2e8c83665f"
format_stamp: "Formatted at 2026-09-29 19:14:27 on dist-test-slave-wftq"
server_key: "e87bd90bdef0861081595cae00d40b62"
server_key_iv: "9b39a20ea970482a7f2090a076e2329f"
server_key_version: "encryptionkey@0"
I20260929 19:14:27.709753 32021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "9af55692ae2a4d5eba662c2e8c83665f"
format_stamp: "Formatted at 2026-09-29 19:14:27 on dist-test-slave-wftq"
server_key: "e87bd90bdef0861081595cae00d40b62"
server_key_iv: "9b39a20ea970482a7f2090a076e2329f"
server_key_version: "encryptionkey@0"
I20260929 19:14:27.711105 32021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:14:27.711930 32036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:27.712134 32021 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:27.712234 32021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "9af55692ae2a4d5eba662c2e8c83665f"
format_stamp: "Formatted at 2026-09-29 19:14:27 on dist-test-slave-wftq"
server_key: "e87bd90bdef0861081595cae00d40b62"
server_key_iv: "9b39a20ea970482a7f2090a076e2329f"
server_key_version: "encryptionkey@0"
I20260929 19:14:27.712299 32021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:27.722961 32021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:27.724778 32021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:27.724924 32021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:27.729552 32021 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:38659
I20260929 19:14:27.729612 32088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:38659 every 8 connection(s)
I20260929 19:14:27.729972 32021 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:27.730897 32089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:27.733170 32089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f: Bootstrap starting.
I20260929 19:14:27.733868 32089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:27.734136 32089 log.cc:824] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:27.734908 32089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f: No bootstrap required, opened a new log
I20260929 19:14:27.736479 32089 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af55692ae2a4d5eba662c2e8c83665f" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38659 } }
I20260929 19:14:27.736622 32089 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:27.736796 32089 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9af55692ae2a4d5eba662c2e8c83665f, State: Initialized, Role: FOLLOWER
I20260929 19:14:27.736927 32089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af55692ae2a4d5eba662c2e8c83665f" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38659 } }
I20260929 19:14:27.737012 32089 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:27.737035 32089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:27.737084 32089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:27.737959 32089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af55692ae2a4d5eba662c2e8c83665f" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38659 } }
I20260929 19:14:27.738049 32089 leader_election.cc:302] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9af55692ae2a4d5eba662c2e8c83665f; no voters:
I20260929 19:14:27.738206 32089 leader_election.cc:288] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:27.738260 32092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:27.738453 32092 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f [term 1 LEADER]: Becoming Leader. State: Replica: 9af55692ae2a4d5eba662c2e8c83665f, State: Running, Role: LEADER
I20260929 19:14:27.738464 32089 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:27.738628 32092 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af55692ae2a4d5eba662c2e8c83665f" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38659 } }
I20260929 19:14:27.738857 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 32021
I20260929 19:14:27.738981 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:27.739084 32093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9af55692ae2a4d5eba662c2e8c83665f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af55692ae2a4d5eba662c2e8c83665f" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38659 } } }
I20260929 19:14:27.739073 32094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9af55692ae2a4d5eba662c2e8c83665f. Latest consensus state: current_term: 1 leader_uuid: "9af55692ae2a4d5eba662c2e8c83665f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af55692ae2a4d5eba662c2e8c83665f" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38659 } } }
I20260929 19:14:27.739181 10160 external_mini_cluster.cc:1428] Setting key c251f321f4daac3aab7376842afe2148
I20260929 19:14:27.739171 32093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:27.739176 32094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:27.740001 32100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:27.740526 32100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:27.742219 32100 catalog_manager.cc:1347] Generated new cluster ID: 0bc5263ff0744da6838c76f2be4178c9
I20260929 19:14:27.742266 32100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 29 19:14:27 dist-test-slave-wftq krb5kdc[32008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:14:27.752259 32101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:27.739651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51049 (local address 127.9.236.62:38659)
0929 19:14:27.739887 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:27.739897 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:27.739931 (+ 34us) server_negotiation.cc:407] Connection header received
0929 19:14:27.740082 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:27.740090 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:27.740155 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:27.740317 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:27.741043 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:27.742497 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:27.743703 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:27.744383 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:27.744896 (+ 513us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:27.747523 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:27.747528 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:27.747528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:27.747542 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:27.750157 (+ 2615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:27.750570 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:27.750573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:27.750573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:27.750705 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:27.750914 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:27.750915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:27.750915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:27.751230 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:27.751764 (+ 534us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:27.751842 (+ 78us) server_negotiation.cc:299] Negotiation successful
0929 19:14:27.751999 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":84,"threads_started":1}
I20260929 19:14:27.753258 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:14:27.762207 32101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:27.755030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53477 (local address 127.9.236.62:38659)
0929 19:14:27.755084 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:27.755087 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:27.755264 (+ 177us) server_negotiation.cc:407] Connection header received
0929 19:14:27.755298 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:27.755299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:27.755310 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:27.755365 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:27.756010 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:27.756807 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:27.757812 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:27.758142 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:27.758214 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:27.759527 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:27.759529 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:27.759529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:27.759536 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:27.761066 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:27.761470 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:27.761471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:27.761471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:27.761527 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:27.761703 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:27.761704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:27.761704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:27.761828 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:27.761949 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:27.762037 (+ 88us) server_negotiation.cc:299] Negotiation successful
0929 19:14:27.762112 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 19:14:27.780318 32100 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:27.780853 32100 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:27.791163 32100 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9af55692ae2a4d5eba662c2e8c83665f: Generated new TSK 0
I20260929 19:14:27.791474 32100 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:27.803735 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 32021
2026-09-29T19:14:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1277 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:27 dist-test-slave-wftq krb5kdc[32120](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:27 dist-test-slave-wftq krb5kdc[32120](info): set up 2 sockets
Sep 29 19:14:27 dist-test-slave-wftq krb5kdc[32120](info): commencing operation
krb5kdc: starting...
W20260929 19:14:28.761308 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:28 dist-test-slave-wftq krb5kdc[32120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:14:28.879833 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:34641
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:33963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:34641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:28.964895 32133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:28.965161 32133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:28.965185 32133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:28.966344 32133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:28.966384 32133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:28.966396 32133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:28.966415 32133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:28.966437 32133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:28.967851 32133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:33963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:34641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:34641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:28.968050 32133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:28.968241 32133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:28.973253 32139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:28.973280 32138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:28.973271 32141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:28.973497 32133 server_base.cc:1034] running on GCE node
I20260929 19:14:28.973760 32133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:28.974054 32133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:28.975198 32133 hybrid_clock.cc:648] HybridClock initialized: now 1790709268975172 us; error 43 us; skew 500 ppm
Sep 29 19:14:28 dist-test-slave-wftq krb5kdc[32120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709268, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:28.977795 32133 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:14:28.978227 32133 webserver.cc:460] Webserver started at http://127.9.236.62:39109/ using document root <none> and password file <none>
I20260929 19:14:28.978430 32133 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:28.978477 32133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:28.978600 32133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:28.979672 32133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "b868081245a14e5cbed62b9b28eff564"
format_stamp: "Formatted at 2026-09-29 19:14:28 on dist-test-slave-wftq"
server_key: "3f7878a6afe5a1a07ae90879035b7bff"
server_key_iv: "c8f0ed73ad5fa8880cfcedae49645340"
server_key_version: "encryptionkey@0"
I20260929 19:14:28.980013 32133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "b868081245a14e5cbed62b9b28eff564"
format_stamp: "Formatted at 2026-09-29 19:14:28 on dist-test-slave-wftq"
server_key: "3f7878a6afe5a1a07ae90879035b7bff"
server_key_iv: "c8f0ed73ad5fa8880cfcedae49645340"
server_key_version: "encryptionkey@0"
I20260929 19:14:28.981353 32133 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 19:14:28.982096 32148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:28.982285 32133 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260929 19:14:28.982367 32133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "b868081245a14e5cbed62b9b28eff564"
format_stamp: "Formatted at 2026-09-29 19:14:28 on dist-test-slave-wftq"
server_key: "3f7878a6afe5a1a07ae90879035b7bff"
server_key_iv: "c8f0ed73ad5fa8880cfcedae49645340"
server_key_version: "encryptionkey@0"
I20260929 19:14:28.982429 32133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:28.998626 32133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:29.000468 32133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:29.000602 32133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:29.004747 32133 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:34641
I20260929 19:14:29.004858 32200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:34641 every 8 connection(s)
I20260929 19:14:29.005160 32133 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:29.006171 32201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:29.008811 32201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564: Bootstrap starting.
I20260929 19:14:29.009521 32201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:29.009821 32201 log.cc:824] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:29.010669 32201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564: No bootstrap required, opened a new log
I20260929 19:14:29.012476 32201 raft_consensus.cc:348] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b868081245a14e5cbed62b9b28eff564" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 34641 } }
I20260929 19:14:29.012645 32201 raft_consensus.cc:374] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:29.012668 32201 raft_consensus.cc:729] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b868081245a14e5cbed62b9b28eff564, State: Initialized, Role: FOLLOWER
I20260929 19:14:29.012837 32201 consensus_queue.cc:260] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b868081245a14e5cbed62b9b28eff564" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 34641 } }
I20260929 19:14:29.012918 32201 raft_consensus.cc:388] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:29.012944 32201 raft_consensus.cc:482] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:29.012995 32201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:29.013950 32201 raft_consensus.cc:504] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b868081245a14e5cbed62b9b28eff564" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 34641 } }
I20260929 19:14:29.014051 32201 leader_election.cc:302] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b868081245a14e5cbed62b9b28eff564; no voters:
I20260929 19:14:29.014247 32201 leader_election.cc:288] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:29.014300 32204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:29.014496 32204 raft_consensus.cc:686] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564 [term 1 LEADER]: Becoming Leader. State: Replica: b868081245a14e5cbed62b9b28eff564, State: Running, Role: LEADER
I20260929 19:14:29.014549 32201 sys_catalog.cc:564] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:29.014621 32204 consensus_queue.cc:237] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b868081245a14e5cbed62b9b28eff564" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 34641 } }
I20260929 19:14:29.014922 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 32133
I20260929 19:14:29.015058 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:29.015161 32205 sys_catalog.cc:455] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b868081245a14e5cbed62b9b28eff564" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b868081245a14e5cbed62b9b28eff564" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 34641 } } }
I20260929 19:14:29.015277 10160 external_mini_cluster.cc:1428] Setting key 1552528c85cf8b8a50c32253297151d5
I20260929 19:14:29.015268 32205 sys_catalog.cc:458] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:29.015446 32206 sys_catalog.cc:455] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b868081245a14e5cbed62b9b28eff564. Latest consensus state: current_term: 1 leader_uuid: "b868081245a14e5cbed62b9b28eff564" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b868081245a14e5cbed62b9b28eff564" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 34641 } } }
I20260929 19:14:29.015589 32206 sys_catalog.cc:458] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:29.015684 32213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:29.016249 32213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:29.018801 32213 catalog_manager.cc:1347] Generated new cluster ID: cb0216ccf23e4a56a818d6b25076c812
I20260929 19:14:29.018858 32213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:29.021130 32217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:29.016021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35513 (local address 127.9.236.62:34641)
0929 19:14:29.016214 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:29.016225 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:29.016258 (+ 33us) server_negotiation.cc:407] Connection header received
0929 19:14:29.016457 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:29.016466 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:29.016545 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:29.016982 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:29.017089 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:29.018263 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:29.019225 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:29.019680 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:29.019775 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:29.020450 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:29.020454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:29.020455 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:29.020640 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:29.020675 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:29.020686 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:29.020742 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:29.020813 (+ 71us) server_negotiation.cc:299] Negotiation successful
0929 19:14:29.020880 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":91,"threads_started":1}
I20260929 19:14:29.022082 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:14:29.026152 32217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:29.023470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57891 (local address 127.9.236.62:34641)
0929 19:14:29.023504 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:29.023506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:29.023622 (+ 116us) server_negotiation.cc:407] Connection header received
0929 19:14:29.023676 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:29.023677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:29.023688 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:29.023732 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:29.023841 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:29.024329 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:29.024994 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:29.025267 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:29.025328 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:29.025931 (+ 603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:29.025932 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:29.025933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:29.025939 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:29.025954 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:29.025958 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:29.026000 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:29.026048 (+ 48us) server_negotiation.cc:299] Negotiation successful
0929 19:14:29.026076 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 19:14:29.036032 32213 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:29.036685 32213 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:29.039995 32213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b868081245a14e5cbed62b9b28eff564: Generated new TSK 0
I20260929 19:14:29.040195 32213 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:29.044337 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 32133
2026-09-29T19:14:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1240 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:29 dist-test-slave-wftq krb5kdc[32232](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:29 dist-test-slave-wftq krb5kdc[32232](info): set up 2 sockets
Sep 29 19:14:29 dist-test-slave-wftq krb5kdc[32232](info): commencing operation
krb5kdc: starting...
W20260929 19:14:29.989389 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:30 dist-test-slave-wftq krb5kdc[32232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:14:30.111963 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:44931
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:46707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:44931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:30.196672 32245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:30.196913 32245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:30.196934 32245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:30.198191 32245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:30.198243 32245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:30.198256 32245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:30.198267 32245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:30.198279 32245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:30.199805 32245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:46707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:44931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:44931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:30.200055 32245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:30.200290 32245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 19:14:30.205554 32245 server_base.cc:1034] running on GCE node
W20260929 19:14:30.205437 32253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:30.205458 32251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:30.205456 32250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:30.206048 32245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:30.206360 32245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:30.207535 32245 hybrid_clock.cc:648] HybridClock initialized: now 1790709270207508 us; error 43 us; skew 500 ppm
Sep 29 19:14:30 dist-test-slave-wftq krb5kdc[32232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709270, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:30.210258 32245 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:14:30.210734 32245 webserver.cc:460] Webserver started at http://127.9.236.62:46405/ using document root <none> and password file <none>
I20260929 19:14:30.210952 32245 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:30.211006 32245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:30.211134 32245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:30.212255 32245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "5ab90c593e134f54b15c79dcc3ce8a9a"
format_stamp: "Formatted at 2026-09-29 19:14:30 on dist-test-slave-wftq"
server_key: "c06c78439d70a3890c90170033e6a9db"
server_key_iv: "d6c90142a486d72979ca277ccf80e77a"
server_key_version: "encryptionkey@0"
I20260929 19:14:30.212599 32245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "5ab90c593e134f54b15c79dcc3ce8a9a"
format_stamp: "Formatted at 2026-09-29 19:14:30 on dist-test-slave-wftq"
server_key: "c06c78439d70a3890c90170033e6a9db"
server_key_iv: "d6c90142a486d72979ca277ccf80e77a"
server_key_version: "encryptionkey@0"
I20260929 19:14:30.213981 32245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:14:30.214805 32260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:30.215021 32245 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260929 19:14:30.215104 32245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "5ab90c593e134f54b15c79dcc3ce8a9a"
format_stamp: "Formatted at 2026-09-29 19:14:30 on dist-test-slave-wftq"
server_key: "c06c78439d70a3890c90170033e6a9db"
server_key_iv: "d6c90142a486d72979ca277ccf80e77a"
server_key_version: "encryptionkey@0"
I20260929 19:14:30.215163 32245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:30.218557 32245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:30.220510 32245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:30.220631 32245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:30.225071 32245 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:44931
I20260929 19:14:30.225167 32312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:44931 every 8 connection(s)
I20260929 19:14:30.225454 32245 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:30.226496 32313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:30.226648 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 32245
I20260929 19:14:30.226760 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:30.226986 10160 external_mini_cluster.cc:1428] Setting key ea465269b75a89a326ba3d2a19cc83f1
I20260929 19:14:30.229683 32316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:30.227508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34315 (local address 127.9.236.62:44931)
0929 19:14:30.227731 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:30.227741 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:30.227778 (+ 37us) server_negotiation.cc:407] Connection header received
0929 19:14:30.228237 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:30.228246 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:30.228279 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:30.228443 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:30.229218 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:30.229222 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:30.229222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:30.229227 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:30.229274 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:30.229294 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:30.229374 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:30.229438 (+ 64us) server_negotiation.cc:299] Negotiation successful
0929 19:14:30.229479 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":92,"threads_started":1}
I20260929 19:14:30.229986 32313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a: Bootstrap starting.
I20260929 19:14:30.230649 32313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:30.230788 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:14:30.230943 32313 log.cc:824] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:30.231750 32313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a: No bootstrap required, opened a new log
I20260929 19:14:30.233451 32316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:30.232457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49449 (local address 127.9.236.62:44931)
0929 19:14:30.232500 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:30.232502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:30.232609 (+ 107us) server_negotiation.cc:407] Connection header received
0929 19:14:30.232654 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:30.232655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:30.232660 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:30.232728 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:30.233235 (+ 507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:30.233236 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:30.233236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 19:14:30.233240 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:30.233255 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 19:14:30.233259 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:30.233338 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:30.233385 (+ 47us) server_negotiation.cc:299] Negotiation successful
0929 19:14:30.233397 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 19:14:30.234076 32313 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ab90c593e134f54b15c79dcc3ce8a9a" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 44931 } }
I20260929 19:14:30.234211 32313 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 19:14:30.234241 32313 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab90c593e134f54b15c79dcc3ce8a9a, State: Initialized, Role: FOLLOWER
I20260929 19:14:30.234364 32313 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ab90c593e134f54b15c79dcc3ce8a9a" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 44931 } }
I20260929 19:14:30.234433 32313 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 19:14:30.234462 32313 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 19:14:30.234508 32313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:30.235028 32313 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ab90c593e134f54b15c79dcc3ce8a9a" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 44931 } }
I20260929 19:14:30.235137 32313 leader_election.cc:302] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ab90c593e134f54b15c79dcc3ce8a9a; no voters:
I20260929 19:14:30.235301 32313 leader_election.cc:288] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 19:14:30.235368 32323 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:30.235517 32323 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a [term 1 LEADER]: Becoming Leader. State: Replica: 5ab90c593e134f54b15c79dcc3ce8a9a, State: Running, Role: LEADER
I20260929 19:14:30.235550 32313 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:30.235622 32323 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ab90c593e134f54b15c79dcc3ce8a9a" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 44931 } }
I20260929 19:14:30.236011 32323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ab90c593e134f54b15c79dcc3ce8a9a. Latest consensus state: current_term: 1 leader_uuid: "5ab90c593e134f54b15c79dcc3ce8a9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ab90c593e134f54b15c79dcc3ce8a9a" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 44931 } } }
I20260929 19:14:30.236021 32325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ab90c593e134f54b15c79dcc3ce8a9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ab90c593e134f54b15c79dcc3ce8a9a" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 44931 } } }
I20260929 19:14:30.236111 32323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:30.236115 32325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:30.236446 32332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:30.237054 32332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:30.238813 32332 catalog_manager.cc:1347] Generated new cluster ID: 21c7de5078914b0c835764b3e936e66d
I20260929 19:14:30.238862 32332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:30.245824 32332 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:30.246012 32332 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:30.252446 32332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ab90c593e134f54b15c79dcc3ce8a9a: Generated new TSK 0
I20260929 19:14:30.252650 32332 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:30.259466 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 32245
2026-09-29T19:14:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1214 ms)
[----------] 18 tests from AuthTokenIssuingTest (22427 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:30 dist-test-slave-wftq krb5kdc[32344](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:30 dist-test-slave-wftq krb5kdc[32344](info): set up 2 sockets
Sep 29 19:14:30 dist-test-slave-wftq krb5kdc[32344](info): commencing operation
krb5kdc: starting...
W20260929 19:14:31.217471 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:31 dist-test-slave-wftq krb5kdc[32344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:14:31.355800 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:36913
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:35139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:36913,127.9.236.61:36089,127.9.236.60:36091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:31.438570 32357 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 19:14:31.438795 32357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:31.438818 32357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:31.438843 32357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:31.439949 32357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:31.439988 32357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:31.440006 32357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:31.440021 32357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:31.440037 32357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:31.441535 32357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:35139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:36913,127.9.236.61:36089,127.9.236.60:36091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:36913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:31.441780 32357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:31.441977 32357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:31.447162 32363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:31.447302 32365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:31.447190 32357 server_base.cc:1034] running on GCE node
W20260929 19:14:31.447192 32362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:31.447754 32357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:31.448117 32357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:31.449267 32357 hybrid_clock.cc:648] HybridClock initialized: now 1790709271449243 us; error 45 us; skew 500 ppm
Sep 29 19:14:31 dist-test-slave-wftq krb5kdc[32344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709271, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:31.452001 32357 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:14:31.452458 32357 webserver.cc:460] Webserver started at http://127.9.236.62:43639/ using document root <none> and password file <none>
I20260929 19:14:31.452656 32357 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:31.452699 32357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:31.452805 32357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:31.454000 32357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "5110fcca9ce049c48662467052181669"
format_stamp: "Formatted at 2026-09-29 19:14:31 on dist-test-slave-wftq"
server_key: "27f67fc1d961261a79ea0031a4a07cc2"
server_key_iv: "c64f3b4b0ee3c740cee5b9e7a0bbf8bc"
server_key_version: "encryptionkey@0"
I20260929 19:14:31.454326 32357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "5110fcca9ce049c48662467052181669"
format_stamp: "Formatted at 2026-09-29 19:14:31 on dist-test-slave-wftq"
server_key: "27f67fc1d961261a79ea0031a4a07cc2"
server_key_iv: "c64f3b4b0ee3c740cee5b9e7a0bbf8bc"
server_key_version: "encryptionkey@0"
I20260929 19:14:31.455785 32357 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:14:31.456612 32372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:31.456849 32357 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:31.456948 32357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "5110fcca9ce049c48662467052181669"
format_stamp: "Formatted at 2026-09-29 19:14:31 on dist-test-slave-wftq"
server_key: "27f67fc1d961261a79ea0031a4a07cc2"
server_key_iv: "c64f3b4b0ee3c740cee5b9e7a0bbf8bc"
server_key_version: "encryptionkey@0"
I20260929 19:14:31.457015 32357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:31.467419 32357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:31.469431 32357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:31.469561 32357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:31.474088 32357 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:36913
I20260929 19:14:31.474157 32424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:36913 every 8 connection(s)
I20260929 19:14:31.474489 32357 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:31.475499 32425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:31.477130 32425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36913 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:14:31.481345 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 32357
I20260929 19:14:31.481424 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:31.481586 10160 external_mini_cluster.cc:1428] Setting key 0ddc55ebf34b0c3053c02a1b8e8a56e8
Sep 29 19:14:31 dist-test-slave-wftq krb5kdc[32344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709271, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:14:31.487300 32428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.477834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55462 (local address 127.9.236.62:36913)
0929 19:14:31.477980 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.477990 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.478161 (+ 171us) server_negotiation.cc:407] Connection header received
0929 19:14:31.478656 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.478658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.478679 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.478781 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.479306 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.480170 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.481042 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.481471 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.481577 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.483944 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:31.483952 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:31.483952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:31.483961 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:31.485633 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.486101 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.486103 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.486103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.486179 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.486426 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.486426 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.486427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.486670 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:31.486735 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:31.486984 (+ 249us) server_negotiation.cc:299] Negotiation successful
0929 19:14:31.487129 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260929 19:14:31.487380 32427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.477777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:36913 (local address 127.0.0.1:55462)
0929 19:14:31.478050 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.478079 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:31.478509 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:31.478805 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:31.478808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:31.479056 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:31.479221 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:31.479226 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.480252 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.480252 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:31.480972 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:31.480974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.481564 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.481566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:31.481639 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.482148 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:31.482156 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:31.483771 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:31.485719 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:31.485728 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:31.485730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:31.486011 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:31.486243 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:31.486244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:31.486244 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:31.486357 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:31.486741 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:31.486747 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:31.486841 (+ 94us) client_negotiation.cc:769] Sending connection context
0929 19:14:31.486955 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 19:14:31.487217 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":74,"thread_start_us":116,"threads_started":1}
I20260929 19:14:31.488127 32425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:14:31.488533 32427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.488391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:36089 (local address 127.0.0.1:34200)
0929 19:14:31.488450 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.488507 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.61:36089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260929 19:14:31.488685 32374 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.236.61:36089: connect: Connection refused (error 111)
I20260929 19:14:31.489140 32427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.489004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:36089 (local address 127.0.0.1:34214)
0929 19:14:31.489062 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.489093 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.61:36089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260929 19:14:31.489260 32425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.61:36089: Network error: Client connection negotiation failed: client connection to 127.9.236.61:36089: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.9.236.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.61.keytab.
Entry for principal kudu/127.9.236.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.61.keytab.
I20260929 19:14:31.508533 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.61:36089
--webserver_interface=127.9.236.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.61.keytab
--principal=kudu/127.9.236.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:35139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:36913,127.9.236.61:36089,127.9.236.60:36091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:14:31.538590 32425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } attempt: 1
I20260929 19:14:31.539364 32427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.539165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:36089 (local address 127.0.0.1:34230)
0929 19:14:31.539258 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.539318 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.61:36089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 19:14:31.539839 32427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.539701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:36089 (local address 127.0.0.1:34244)
0929 19:14:31.539778 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.539815 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.61:36089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260929 19:14:31.539959 32425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.61:36089: Network error: Client connection negotiation failed: client connection to 127.9.236.61:36089: connect: Connection refused (error 111)
W20260929 19:14:31.597364 32432 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 19:14:31.597632 32432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:31.597657 32432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:31.597683 32432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:31.598896 32432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:31.598935 32432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:31.598953 32432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:31.598970 32432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:31.598984 32432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:31.600648 32432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:35139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:36913,127.9.236.61:36089,127.9.236.60:36091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.61.keytab
--principal=kudu/127.9.236.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.61:36089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.236.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:31.600947 32432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:31.601208 32432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:31.606288 32438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:31.606504 32432 server_base.cc:1034] running on GCE node
W20260929 19:14:31.606274 32440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:31.606274 32437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:31.606894 32432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:31.607219 32432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:31.608114 32425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } attempt: 2
I20260929 19:14:31.608388 32432 hybrid_clock.cc:648] HybridClock initialized: now 1790709271608367 us; error 38 us; skew 500 ppm
I20260929 19:14:31.608604 32427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.608477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:36089 (local address 127.0.0.1:34248)
0929 19:14:31.608530 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.608565 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.61:36089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260929 19:14:31.608995 32427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.608841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:36089 (local address 127.0.0.1:34264)
0929 19:14:31.608912 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.608960 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.61:36089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260929 19:14:31.609156 32425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.61:36089: Network error: Client connection negotiation failed: client connection to 127.9.236.61:36089: connect: Connection refused (error 111)
Sep 29 19:14:31 dist-test-slave-wftq krb5kdc[32344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709271, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:31.611202 32432 init.cc:377] Logged in from keytab as kudu/127.9.236.61@KRBTEST.COM (short username kudu)
I20260929 19:14:31.611722 32432 webserver.cc:460] Webserver started at http://127.9.236.61:37575/ using document root <none> and password file <none>
I20260929 19:14:31.611958 32432 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:31.612010 32432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:31.612124 32432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:31.613291 32432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-1/data/instance:
uuid: "c2c8e149cc71401c88376ba1e0f4bc7b"
format_stamp: "Formatted at 2026-09-29 19:14:31 on dist-test-slave-wftq"
server_key: "72a0830425a6d5bff6f402e9c065c1e5"
server_key_iv: "7c9bf47ba9a51f1e00f460e4fd2bed68"
server_key_version: "encryptionkey@0"
I20260929 19:14:31.613654 32432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-1/wal/instance:
uuid: "c2c8e149cc71401c88376ba1e0f4bc7b"
format_stamp: "Formatted at 2026-09-29 19:14:31 on dist-test-slave-wftq"
server_key: "72a0830425a6d5bff6f402e9c065c1e5"
server_key_iv: "7c9bf47ba9a51f1e00f460e4fd2bed68"
server_key_version: "encryptionkey@0"
I20260929 19:14:31.614936 32432 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:14:31.615797 32447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:31.616037 32432 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:31.616142 32432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-1/wal
uuid: "c2c8e149cc71401c88376ba1e0f4bc7b"
format_stamp: "Formatted at 2026-09-29 19:14:31 on dist-test-slave-wftq"
server_key: "72a0830425a6d5bff6f402e9c065c1e5"
server_key_iv: "7c9bf47ba9a51f1e00f460e4fd2bed68"
server_key_version: "encryptionkey@0"
I20260929 19:14:31.616204 32432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:31.626813 32432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:31.628635 32432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:31.628770 32432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:31.632974 32432 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.61:36089
I20260929 19:14:31.633035 32499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.61:36089 every 8 connection(s)
I20260929 19:14:31.633404 32432 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-1/data/info.pb
I20260929 19:14:31.633842 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 32432
I20260929 19:14:31.633921 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-1/wal/instance
I20260929 19:14:31.634095 10160 external_mini_cluster.cc:1428] Setting key 588aa92e0f8cff95dcde28c3ea4febcf
I20260929 19:14:31.634568 32500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:31.636325 32500 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36913 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.236.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.60@KRBTEST.COM" created.
Sep 29 19:14:31 dist-test-slave-wftq krb5kdc[32344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709271, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.61@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:14:31.649508 32428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.637154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55464 (local address 127.9.236.62:36913)
0929 19:14:31.637232 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.637235 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.637492 (+ 257us) server_negotiation.cc:407] Connection header received
0929 19:14:31.638178 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.638180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.638194 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.638258 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.639202 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.639706 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.641463 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.641673 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.641738 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.646562 (+ 4824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:31.646564 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:31.646564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:31.646573 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:31.648046 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.648540 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.648540 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.648541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.648586 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.648866 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.648867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.648867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.648971 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:31.649016 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:31.649373 (+ 357us) server_negotiation.cc:299] Negotiation successful
0929 19:14:31.649440 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 19:14:31.649770 32503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.637085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:36913 (local address 127.0.0.1:55464)
0929 19:14:31.637327 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.637379 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:31.638065 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:31.638350 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:31.638356 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:31.638751 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:31.639071 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:31.639077 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.639810 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.639811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:31.641355 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:31.641358 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.641786 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.641787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:31.641942 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.642516 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:31.642528 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:31.646447 (+ 3919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:31.648133 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:31.648142 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:31.648146 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:31.648464 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:31.648666 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:31.648667 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:31.648667 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:31.648804 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:31.649023 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:31.649028 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:31.649227 (+ 199us) client_negotiation.cc:769] Sending connection context
0929 19:14:31.649343 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 19:14:31.649511 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":99,"threads_started":1}
I20260929 19:14:31.650439 32500 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 19:14:31 dist-test-slave-wftq krb5kdc[32344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709271, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.61@KRBTEST.COM for kudu/127.9.236.61@KRBTEST.COM
I20260929 19:14:31.660473 32503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.650734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:36089 (local address 127.0.0.1:34266)
0929 19:14:31.650799 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.650811 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:31.650854 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:31.651136 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:31.651138 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:31.651291 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:31.651360 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:31.651362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.652318 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.652319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:31.653183 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:31.653186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.653672 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.653673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:31.653758 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.654205 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:31.654211 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:31.657064 (+ 2853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:31.659049 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:31.659050 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:31.659051 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:31.659313 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:31.659560 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:31.659561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:31.659562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:31.659630 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:31.660065 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:31.660067 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:31.660137 (+ 70us) client_negotiation.cc:769] Sending connection context
0929 19:14:31.660227 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 19:14:31.660365 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 19:14:31.660605 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.650819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34266 (local address 127.9.236.61:36089)
0929 19:14:31.650986 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.650992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.651012 (+ 20us) server_negotiation.cc:407] Connection header received
0929 19:14:31.651033 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.651035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.651045 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.651110 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.651446 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.652230 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.653263 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.653585 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.653652 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.657196 (+ 3544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:31.657205 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:31.657206 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:31.657213 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:31.658920 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.659407 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.659411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.659411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.659482 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.659701 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.659703 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.659703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.659978 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:31.660072 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:31.660300 (+ 228us) server_negotiation.cc:299] Negotiation successful
0929 19:14:31.660442 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":130,"thread_start_us":103,"threads_started":1}
I20260929 19:14:31.661375 32500 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:14:31.661747 32503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.661631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:36091 (local address 127.0.0.1:37068)
0929 19:14:31.661676 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.661724 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:36091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 19:14:31.661902 32449 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.236.60:36091: connect: Connection refused (error 111)
I20260929 19:14:31.662438 32503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.662320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:36091 (local address 127.0.0.1:37076)
0929 19:14:31.662362 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.662416 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:36091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260929 19:14:31.662608 32500 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.60:36091: Network error: Client connection negotiation failed: client connection to 127.9.236.60:36091: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.60.keytab.
Entry for principal kudu/127.9.236.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.60.keytab.
I20260929 19:14:31.663870 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.60:36091
--webserver_interface=127.9.236.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.60.keytab
--principal=kudu/127.9.236.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:35139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:36913,127.9.236.61:36089,127.9.236.60:36091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:14:31.700340 32425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } attempt: 3
Sep 29 19:14:31 dist-test-slave-wftq krb5kdc[32344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709271, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for kudu/127.9.236.61@KRBTEST.COM
I20260929 19:14:31.707811 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.700931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34272 (local address 127.9.236.61:36089)
0929 19:14:31.701019 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.701022 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.701031 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:14:31.701063 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.701064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.701077 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.701171 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.701531 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.702075 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.702778 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.703011 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.703070 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.705417 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:31.705418 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:31.705419 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:31.705427 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:31.706740 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.707083 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.707084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.707084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.707134 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.707291 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.707298 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.707298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.707414 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:31.707470 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:31.707608 (+ 138us) server_negotiation.cc:299] Negotiation successful
0929 19:14:31.707686 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 19:14:31.707825 32427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.700831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:36089 (local address 127.0.0.1:34272)
0929 19:14:31.700908 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.700921 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:31.700969 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:31.701180 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:31.701181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:31.701366 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:31.701450 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:31.701452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.702151 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.702152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:31.702699 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:31.702701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.703073 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.703073 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:31.703147 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.703645 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:31.703650 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:31.705331 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:31.706862 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:31.706863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:31.706864 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:31.707021 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:31.707187 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:31.707187 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:31.707188 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:31.707244 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:31.707466 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:31.707467 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:31.707519 (+ 52us) client_negotiation.cc:769] Sending connection context
0929 19:14:31.707579 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 19:14:31.707686 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260929 19:14:31.709343 32425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:14:31.709690 32427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.709595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:36091 (local address 127.0.0.1:37090)
0929 19:14:31.709637 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.709665 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:36091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260929 19:14:31.710095 32427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.710003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:36091 (local address 127.0.0.1:37106)
0929 19:14:31.710059 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.710082 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:36091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 19:14:31.710227 32425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.60:36091: Network error: Client connection negotiation failed: client connection to 127.9.236.60:36091: connect: Connection refused (error 111)
I20260929 19:14:31.711856 32500 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } attempt: 1
I20260929 19:14:31.712309 32503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.712164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:36091 (local address 127.0.0.1:37112)
0929 19:14:31.712234 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.712279 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:36091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 19:14:31.712634 32503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.712542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:36091 (local address 127.0.0.1:37116)
0929 19:14:31.712593 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.712619 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:36091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260929 19:14:31.712755 32500 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.60:36091: Network error: Client connection negotiation failed: client connection to 127.9.236.60:36091: connect: Connection refused (error 111)
I20260929 19:14:31.741376 32425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } attempt: 1
I20260929 19:14:31.741930 32427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.741778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:36091 (local address 127.0.0.1:37120)
0929 19:14:31.741854 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.741894 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:36091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260929 19:14:31.742455 32427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.742292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:36091 (local address 127.0.0.1:37132)
0929 19:14:31.742384 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.742430 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:36091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260929 19:14:31.742640 32425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.60:36091: Network error: Client connection negotiation failed: client connection to 127.9.236.60:36091: connect: Connection refused (error 111)
W20260929 19:14:31.749315 32507 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 19:14:31.749524 32507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:31.749547 32507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:31.749575 32507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:31.750695 32507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:31.750737 32507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:31.750756 32507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:31.750770 32507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:31.750787 32507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:31.752255 32507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:35139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:36913,127.9.236.61:36089,127.9.236.60:36091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.60.keytab
--principal=kudu/127.9.236.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.60:36091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.236.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:31.752486 32507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:31.752686 32507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:31.757468 32515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:31.757511 32512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:31.757485 32513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:31.757870 32507 server_base.cc:1034] running on GCE node
I20260929 19:14:31.758145 32507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:31.758495 32507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:31.759656 32507 hybrid_clock.cc:648] HybridClock initialized: now 1790709271759634 us; error 42 us; skew 500 ppm
Sep 29 19:14:31 dist-test-slave-wftq krb5kdc[32344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709271, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:31.762480 32507 init.cc:377] Logged in from keytab as kudu/127.9.236.60@KRBTEST.COM (short username kudu)
I20260929 19:14:31.763001 32507 webserver.cc:460] Webserver started at http://127.9.236.60:40947/ using document root <none> and password file <none>
I20260929 19:14:31.763237 32507 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:31.763288 32507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:31.763417 32507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:31.764592 32507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-2/data/instance:
uuid: "c57b0f9c5f474f2cb2d8985ad762fc73"
format_stamp: "Formatted at 2026-09-29 19:14:31 on dist-test-slave-wftq"
server_key: "ce937a1f6d7cec4b95ab43c0209445b8"
server_key_iv: "5d4ee9e7362b1ed9134739c206a86bcf"
server_key_version: "encryptionkey@0"
I20260929 19:14:31.764951 32507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-2/wal/instance:
uuid: "c57b0f9c5f474f2cb2d8985ad762fc73"
format_stamp: "Formatted at 2026-09-29 19:14:31 on dist-test-slave-wftq"
server_key: "ce937a1f6d7cec4b95ab43c0209445b8"
server_key_iv: "5d4ee9e7362b1ed9134739c206a86bcf"
server_key_version: "encryptionkey@0"
I20260929 19:14:31.766408 32507 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:31.767282 32522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:31.767508 32507 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:31.767604 32507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-2/wal
uuid: "c57b0f9c5f474f2cb2d8985ad762fc73"
format_stamp: "Formatted at 2026-09-29 19:14:31 on dist-test-slave-wftq"
server_key: "ce937a1f6d7cec4b95ab43c0209445b8"
server_key_iv: "5d4ee9e7362b1ed9134739c206a86bcf"
server_key_version: "encryptionkey@0"
I20260929 19:14:31.767671 32507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:31.777926 32507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:31.780053 32507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:31.780195 32507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:31.780962 32500 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } attempt: 2
I20260929 19:14:31.781502 32503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.781366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:36091 (local address 127.0.0.1:37136)
0929 19:14:31.781424 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.781460 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:36091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 19:14:31.782027 32503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.781873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:36091 (local address 127.0.0.1:37152)
0929 19:14:31.781965 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.782004 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:36091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260929 19:14:31.782202 32500 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.60:36091: Network error: Client connection negotiation failed: client connection to 127.9.236.60:36091: connect: Connection refused (error 111)
I20260929 19:14:31.784927 32507 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.60:36091
I20260929 19:14:31.784978 32574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.60:36091 every 8 connection(s)
I20260929 19:14:31.785337 32507 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-2/data/info.pb
I20260929 19:14:31.786368 32575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:31.787976 32575 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36913 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:14:31.789132 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 32507
I20260929 19:14:31.789227 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790709008521303-10160-0/minicluster-data/master-2/wal/instance
I20260929 19:14:31.789430 10160 external_mini_cluster.cc:1428] Setting key e4b950354756c661bf8169ea0abe6f92
Sep 29 19:14:31 dist-test-slave-wftq krb5kdc[32344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709271, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.60@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
Sep 29 19:14:31 dist-test-slave-wftq krb5kdc[32344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.60@KRBTEST.COM
I20260929 19:14:31.798014 32428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.788665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55478 (local address 127.9.236.62:36913)
0929 19:14:31.788720 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.788721 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.788986 (+ 265us) server_negotiation.cc:407] Connection header received
0929 19:14:31.789567 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.789570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.789585 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.789658 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.790414 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.790944 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.791972 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.792162 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.792215 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.794744 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:31.794746 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:31.794746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:31.794754 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:31.796472 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.796941 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.796943 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.796943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.797011 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.797347 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.797348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.797348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.797466 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:31.797523 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:31.797860 (+ 337us) server_negotiation.cc:299] Negotiation successful
0929 19:14:31.797936 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260929 19:14:31.798233 32577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.788640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:36913 (local address 127.0.0.1:55478)
0929 19:14:31.788841 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.788878 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:31.789434 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:31.789758 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:31.789763 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:31.790103 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:31.790310 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:31.790317 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.791032 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.791033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:31.791884 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:31.791887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.792252 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.792253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:31.792397 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.793064 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:31.793073 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:31.794631 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:31.796578 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:31.796585 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:31.796587 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:31.796865 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:31.797086 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:31.797087 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:31.797087 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:31.797262 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:31.797534 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:31.797538 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:31.797693 (+ 155us) client_negotiation.cc:769] Sending connection context
0929 19:14:31.797808 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 19:14:31.797986 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":20,"thread_start_us":98,"threads_started":1}
I20260929 19:14:31.798735 32575 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:14:31.799290 32579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.789866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37160 (local address 127.9.236.60:36091)
0929 19:14:31.790192 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.790197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.790211 (+ 14us) server_negotiation.cc:407] Connection header received
0929 19:14:31.790232 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.790233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.790251 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.790330 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.790960 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.791863 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.792825 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.793279 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.793349 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.795957 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:31.795966 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:31.795966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:31.795975 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:31.797994 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.798431 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.798435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.798436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.798501 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.798661 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.798662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.798662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.798820 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:31.799010 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:31.799095 (+ 85us) server_negotiation.cc:299] Negotiation successful
0929 19:14:31.799186 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":298,"thread_start_us":93,"threads_started":1}
Sep 29 19:14:31 dist-test-slave-wftq krb5kdc[32344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709271, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.60@KRBTEST.COM for kudu/127.9.236.61@KRBTEST.COM
Sep 29 19:14:31 dist-test-slave-wftq krb5kdc[32344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:14:31.806041 32577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.798983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:36089 (local address 127.0.0.1:34284)
0929 19:14:31.799024 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.799033 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:31.799067 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:31.799192 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:31.799193 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:31.799340 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:31.799401 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:31.799403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.800171 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.800172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:31.800757 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:31.800759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.801193 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.801195 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:31.801270 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.802238 (+ 968us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:31.802243 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:31.803547 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:31.805030 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:31.805031 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:31.805032 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:31.805245 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:31.805450 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:31.805450 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:31.805451 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:31.805511 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:31.805744 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:31.805745 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:31.805792 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 19:14:31.805855 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 19:14:31.805942 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260929 19:14:31.806047 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.799049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34284 (local address 127.9.236.61:36089)
0929 19:14:31.799094 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.799096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.799104 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:14:31.799129 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.799130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.799139 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.799181 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.799509 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.800098 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.800837 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.801119 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.801180 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.803634 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:31.803635 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:31.803636 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:31.803643 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:31.804966 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.805315 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.805316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.805316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.805383 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.805574 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.805574 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.805574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.805679 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:31.805732 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:31.805897 (+ 165us) server_negotiation.cc:299] Negotiation successful
0929 19:14:31.805976 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 19:14:31.806609 32575 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:14:31.807824 32428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.800171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55480 (local address 127.9.236.62:36913)
0929 19:14:31.800235 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.800237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.800272 (+ 35us) server_negotiation.cc:407] Connection header received
0929 19:14:31.800344 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.800346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.800356 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.800409 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.800910 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.801512 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.802190 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.802450 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.802519 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.804973 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:31.804974 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:31.804974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:31.804981 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:31.806687 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.807031 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.807032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.807033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.807104 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.807288 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.807289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.807289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.807409 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:31.807604 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:31.807644 (+ 40us) server_negotiation.cc:299] Negotiation successful
0929 19:14:31.807726 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Sep 29 19:14:31 dist-test-slave-wftq krb5kdc[32344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709271, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.60@KRBTEST.COM for kudu/127.9.236.60@KRBTEST.COM
Sep 29 19:14:31 dist-test-slave-wftq krb5kdc[32344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.61@KRBTEST.COM
I20260929 19:14:31.813640 32579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.806928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37170 (local address 127.9.236.60:36091)
0929 19:14:31.806965 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.806966 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.807003 (+ 37us) server_negotiation.cc:407] Connection header received
0929 19:14:31.807069 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.807070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.807082 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.807135 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.807393 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.808099 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.809068 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.809397 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.809462 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.811229 (+ 1767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:31.811230 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:31.811231 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:31.811236 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:31.812486 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.812755 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.812756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.812756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.812807 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.812956 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.812956 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.812957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.813075 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:31.813172 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:31.813351 (+ 179us) server_negotiation.cc:299] Negotiation successful
0929 19:14:31.813476 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":16}
I20260929 19:14:31.813704 32577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.806886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:36091 (local address 127.0.0.1:37170)
0929 19:14:31.806947 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.806961 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:31.807015 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:31.807147 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:31.807148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:31.807276 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:31.807317 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:31.807318 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.808177 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.808178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:31.808989 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:31.808992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.809466 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.809468 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:31.809520 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.809888 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:31.809892 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:31.811144 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:31.812560 (+ 1416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:31.812561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:31.812561 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:31.812699 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:31.812861 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:31.812862 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:31.812862 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:31.812911 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:31.813177 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:31.813178 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:31.813234 (+ 56us) client_negotiation.cc:769] Sending connection context
0929 19:14:31.813314 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 19:14:31.813432 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 19:14:31.815299 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.808596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34294 (local address 127.9.236.61:36089)
0929 19:14:31.808630 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.808631 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.808641 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:14:31.808675 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.808676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.808685 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.808739 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.809304 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.809846 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.810536 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.810742 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.810797 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.812829 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:31.812830 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:31.812831 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:31.812839 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:31.814262 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.814587 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.814588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.814588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.814643 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.814833 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.814834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.814834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.814968 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:31.815091 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:31.815172 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 19:14:31.815236 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260929 19:14:31.815558 32575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c57b0f9c5f474f2cb2d8985ad762fc73: Bootstrap starting.
I20260929 19:14:31.816476 32575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c57b0f9c5f474f2cb2d8985ad762fc73: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:31.816728 32575 log.cc:824] T 00000000000000000000000000000000 P c57b0f9c5f474f2cb2d8985ad762fc73: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:31.816733 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:14:31.817438 32575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c57b0f9c5f474f2cb2d8985ad762fc73: No bootstrap required, opened a new log
I20260929 19:14:31.817787 32425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } attempt: 2
I20260929 19:14:31.818043 32399 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5110fcca9ce049c48662467052181669"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55480
I20260929 19:14:31.819425 32575 raft_consensus.cc:348] T 00000000000000000000000000000000 P c57b0f9c5f474f2cb2d8985ad762fc73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5110fcca9ce049c48662467052181669" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36913 } } peers { permanent_uuid: "c2c8e149cc71401c88376ba1e0f4bc7b" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } } peers { permanent_uuid: "c57b0f9c5f474f2cb2d8985ad762fc73" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } }
I20260929 19:14:31.819571 32575 raft_consensus.cc:729] T 00000000000000000000000000000000 P c57b0f9c5f474f2cb2d8985ad762fc73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c57b0f9c5f474f2cb2d8985ad762fc73, State: Initialized, Role: FOLLOWER
I20260929 19:14:31.819645 32575 consensus_queue.cc:260] T 00000000000000000000000000000000 P c57b0f9c5f474f2cb2d8985ad762fc73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5110fcca9ce049c48662467052181669" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36913 } } peers { permanent_uuid: "c2c8e149cc71401c88376ba1e0f4bc7b" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } } peers { permanent_uuid: "c57b0f9c5f474f2cb2d8985ad762fc73" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } }
I20260929 19:14:31.819937 32575 sys_catalog.cc:564] T 00000000000000000000000000000000 P c57b0f9c5f474f2cb2d8985ad762fc73 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:31.819988 32581 sys_catalog.cc:455] T 00000000000000000000000000000000 P c57b0f9c5f474f2cb2d8985ad762fc73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5110fcca9ce049c48662467052181669" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36913 } } peers { permanent_uuid: "c2c8e149cc71401c88376ba1e0f4bc7b" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } } peers { permanent_uuid: "c57b0f9c5f474f2cb2d8985ad762fc73" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } } }
I20260929 19:14:31.820112 32581 sys_catalog.cc:458] T 00000000000000000000000000000000 P c57b0f9c5f474f2cb2d8985ad762fc73 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 19:14:31.821951 32592 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c57b0f9c5f474f2cb2d8985ad762fc73: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:14:31.822057 32592 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:14:31.822108 32507 master_runner.cc:186] Error getting master registration for 127.9.236.62:36913: OK, instance_id { permanent_uuid: "5110fcca9ce049c48662467052181669" instance_seqno: 1790709271472783 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
Sep 29 19:14:31 dist-test-slave-wftq krb5kdc[32344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709271, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for kudu/127.9.236.60@KRBTEST.COM
I20260929 19:14:31.824813 32507 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 19:14:31.827418 32427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.818046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:36091 (local address 127.0.0.1:37184)
0929 19:14:31.818097 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.818108 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:31.818142 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:31.821427 (+ 3285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:31.821428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:31.821605 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:31.821672 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:31.821674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.822469 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.822470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:31.822985 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:31.822987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.823302 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.823303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:31.823355 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.823770 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:31.823781 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:31.825131 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:31.826507 (+ 1376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:31.826507 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:31.826508 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:31.826666 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:31.826822 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:31.826822 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:31.826822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:31.826884 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:31.827116 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:31.827117 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:31.827161 (+ 44us) client_negotiation.cc:769] Sending connection context
0929 19:14:31.827221 (+ 60us) client_negotiation.cc:240] Negotiation successful
0929 19:14:31.827319 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260929 19:14:31.827417 32579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.818113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37184 (local address 127.9.236.60:36091)
0929 19:14:31.821294 (+ 3181us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.821296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.821307 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:14:31.821329 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.821330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.821341 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.821414 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.821767 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.822394 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.823054 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.823248 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.823301 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.825200 (+ 1899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:31.825202 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:31.825202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:31.825209 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:31.826433 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.826724 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.826725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.826725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.826772 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.826931 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.826932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.826932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.827060 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:31.827117 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:31.827248 (+ 131us) server_negotiation.cc:299] Negotiation successful
0929 19:14:31.827319 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3157}
I20260929 19:14:31.828967 32425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669: Bootstrap starting.
I20260929 19:14:31.829792 32425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:31.830142 32425 log.cc:824] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:31.830900 32425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669: No bootstrap required, opened a new log
I20260929 19:14:31.832023 32425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5110fcca9ce049c48662467052181669" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36913 } } peers { permanent_uuid: "c2c8e149cc71401c88376ba1e0f4bc7b" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } } peers { permanent_uuid: "c57b0f9c5f474f2cb2d8985ad762fc73" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } }
I20260929 19:14:31.832149 32425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5110fcca9ce049c48662467052181669, State: Initialized, Role: FOLLOWER
I20260929 19:14:31.832229 32425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5110fcca9ce049c48662467052181669" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36913 } } peers { permanent_uuid: "c2c8e149cc71401c88376ba1e0f4bc7b" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } } peers { permanent_uuid: "c57b0f9c5f474f2cb2d8985ad762fc73" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } }
I20260929 19:14:31.832504 32595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5110fcca9ce049c48662467052181669" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36913 } } peers { permanent_uuid: "c2c8e149cc71401c88376ba1e0f4bc7b" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } } peers { permanent_uuid: "c57b0f9c5f474f2cb2d8985ad762fc73" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } } }
I20260929 19:14:31.832578 32425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:31.832587 32595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:14:31.832736 32399 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 19:14:31.832881 32399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:31.834041 32399 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5110fcca9ce049c48662467052181669" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36913 } } peers { permanent_uuid: "c2c8e149cc71401c88376ba1e0f4bc7b" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } } peers { permanent_uuid: "c57b0f9c5f474f2cb2d8985ad762fc73" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } }
I20260929 19:14:31.834947 32549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5110fcca9ce049c48662467052181669" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c57b0f9c5f474f2cb2d8985ad762fc73"
I20260929 19:14:31.835101 32549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c57b0f9c5f474f2cb2d8985ad762fc73 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:31.835929 32549 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c57b0f9c5f474f2cb2d8985ad762fc73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5110fcca9ce049c48662467052181669 in term 1.
I20260929 19:14:31.836149 32374 leader_election.cc:302] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5110fcca9ce049c48662467052181669, c57b0f9c5f474f2cb2d8985ad762fc73; no voters:
I20260929 19:14:31.834355 32399 leader_election.cc:288] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [CANDIDATE]: Term 1 election: Requested vote from peers c2c8e149cc71401c88376ba1e0f4bc7b (127.9.236.61:36089), c57b0f9c5f474f2cb2d8985ad762fc73 (127.9.236.60:36091)
I20260929 19:14:31.836277 32595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:31.836452 32595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [term 1 LEADER]: Becoming Leader. State: Replica: 5110fcca9ce049c48662467052181669, State: Running, Role: LEADER
I20260929 19:14:31.836547 32595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5110fcca9ce049c48662467052181669" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36913 } } peers { permanent_uuid: "c2c8e149cc71401c88376ba1e0f4bc7b" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } } peers { permanent_uuid: "c57b0f9c5f474f2cb2d8985ad762fc73" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } }
I20260929 19:14:31.836977 32595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5110fcca9ce049c48662467052181669. Latest consensus state: current_term: 1 leader_uuid: "5110fcca9ce049c48662467052181669" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5110fcca9ce049c48662467052181669" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36913 } } peers { permanent_uuid: "c2c8e149cc71401c88376ba1e0f4bc7b" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } } peers { permanent_uuid: "c57b0f9c5f474f2cb2d8985ad762fc73" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } } }
I20260929 19:14:31.837057 32595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [sys.catalog]: This master's current role is: LEADER
W20260929 19:14:31.835260 32606 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:14:31.837776 32606 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:14:31.838192 32612 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:31.838420 32612 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:31.838114 32357 master_runner.cc:186] Error getting master registration for 127.9.236.61:36089: OK, instance_id { permanent_uuid: "c2c8e149cc71401c88376ba1e0f4bc7b" instance_seqno: 1790709271631835 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 19:14:31.840540 32357 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 19:14:31.848799 32579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.838724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37190 (local address 127.9.236.60:36091)
0929 19:14:31.838768 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.838770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.839108 (+ 338us) server_negotiation.cc:407] Connection header received
0929 19:14:31.839292 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.839294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.839306 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.839335 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.840126 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.840821 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.843399 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.843694 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.843751 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.845449 (+ 1698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:31.845451 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:31.845451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:31.845458 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:31.847459 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.847813 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.847814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.847815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.847873 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.848058 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.848059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.848059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.848193 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:31.848305 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:31.848622 (+ 317us) server_negotiation.cc:299] Negotiation successful
0929 19:14:31.848691 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:14:31.850754 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.840634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34302 (local address 127.9.236.61:36089)
0929 19:14:31.840687 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.840689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.840699 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:14:31.840722 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.840723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.840734 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.840796 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.841680 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.842415 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.843387 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.844203 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.844265 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.846333 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:31.846334 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:31.846334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:31.846342 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:31.848895 (+ 2553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.849536 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.849537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.849537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.849603 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.849816 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.849818 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.849818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.849969 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:31.850517 (+ 548us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:31.850570 (+ 53us) server_negotiation.cc:299] Negotiation successful
0929 19:14:31.850646 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 19:14:31.852586 32549 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c57b0f9c5f474f2cb2d8985ad762fc73 [term 1 FOLLOWER]: Refusing update from remote peer 5110fcca9ce049c48662467052181669: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:14:31.852958 32608 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c57b0f9c5f474f2cb2d8985ad762fc73" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:14:31.854393 32428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.839144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55482 (local address 127.9.236.62:36913)
0929 19:14:31.839186 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.839187 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.840217 (+ 1030us) server_negotiation.cc:407] Connection header received
0929 19:14:31.840307 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.840308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.840318 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.840356 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.846181 (+ 5825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.846867 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.849465 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.849751 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.849793 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.851086 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:31.851087 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:31.851088 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:31.851095 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:31.852810 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.853586 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.853588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.853588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.853642 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.853823 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.853823 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.853824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.853948 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:31.854038 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:31.854221 (+ 183us) server_negotiation.cc:299] Negotiation successful
0929 19:14:31.854297 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 19:14:31.854645 32581 sys_catalog.cc:455] T 00000000000000000000000000000000 P c57b0f9c5f474f2cb2d8985ad762fc73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5110fcca9ce049c48662467052181669. Latest consensus state: current_term: 1 leader_uuid: "5110fcca9ce049c48662467052181669" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5110fcca9ce049c48662467052181669" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36913 } } peers { permanent_uuid: "c2c8e149cc71401c88376ba1e0f4bc7b" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } } peers { permanent_uuid: "c57b0f9c5f474f2cb2d8985ad762fc73" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } } }
I20260929 19:14:31.854725 32581 sys_catalog.cc:458] T 00000000000000000000000000000000 P c57b0f9c5f474f2cb2d8985ad762fc73 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:14:31.856393 32608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5110fcca9ce049c48662467052181669" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5110fcca9ce049c48662467052181669" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36913 } } peers { permanent_uuid: "c2c8e149cc71401c88376ba1e0f4bc7b" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } } peers { permanent_uuid: "c57b0f9c5f474f2cb2d8985ad762fc73" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } } }
I20260929 19:14:31.856495 32608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:31.857023 32581 sys_catalog.cc:455] T 00000000000000000000000000000000 P c57b0f9c5f474f2cb2d8985ad762fc73 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5110fcca9ce049c48662467052181669" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5110fcca9ce049c48662467052181669" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36913 } } peers { permanent_uuid: "c2c8e149cc71401c88376ba1e0f4bc7b" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } } peers { permanent_uuid: "c57b0f9c5f474f2cb2d8985ad762fc73" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } } }
I20260929 19:14:31.857137 32581 sys_catalog.cc:458] T 00000000000000000000000000000000 P c57b0f9c5f474f2cb2d8985ad762fc73 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:14:31.857370 32612 catalog_manager.cc:1347] Generated new cluster ID: be4e24a03c644d8daa33bff1d81e5865
I20260929 19:14:31.857419 32612 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:31.873359 32500 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } attempt: 3
Sep 29 19:14:31 dist-test-slave-wftq krb5kdc[32344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709271, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.61@KRBTEST.COM for kudu/127.9.236.60@KRBTEST.COM
I20260929 19:14:31.880196 32612 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:31.880975 32612 catalog_manager.cc:1504] Loading token signing keys...
I20260929 19:14:31.882321 32503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.873719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:36091 (local address 127.0.0.1:37192)
0929 19:14:31.873902 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:31.873914 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:31.873953 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:31.874219 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:31.874220 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:31.874421 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:31.874476 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:31.874478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.875389 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.875390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:31.876205 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:31.876206 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.876851 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.876852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:31.876921 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.877525 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:31.877530 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:31.879245 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:31.881215 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:31.881216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:31.881216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:31.881446 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:31.881652 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:31.881653 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:31.881653 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:31.881709 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:31.881993 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:31.881994 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:31.882044 (+ 50us) client_negotiation.cc:769] Sending connection context
0929 19:14:31.882102 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 19:14:31.882213 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160}
I20260929 19:14:31.882679 32579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:31.874054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37192 (local address 127.9.236.60:36091)
0929 19:14:31.874094 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.874095 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.874102 (+ 7us) server_negotiation.cc:407] Connection header received
0929 19:14:31.874122 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.874123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.874132 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.874171 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.874538 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.875320 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.876400 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:31.876709 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:31.876758 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:31.879384 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:31.879386 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:31.879386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:31.879396 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:31.881074 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.881522 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.881523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.881523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.881574 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:31.881767 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:31.881767 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:31.881768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:31.881906 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:31.881966 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:31.882501 (+ 535us) server_negotiation.cc:299] Negotiation successful
0929 19:14:31.882576 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:14:31.884338 32500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2c8e149cc71401c88376ba1e0f4bc7b: Bootstrap starting.
I20260929 19:14:31.885075 32500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2c8e149cc71401c88376ba1e0f4bc7b: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:31.885406 32500 log.cc:824] T 00000000000000000000000000000000 P c2c8e149cc71401c88376ba1e0f4bc7b: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:31.886261 32500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2c8e149cc71401c88376ba1e0f4bc7b: No bootstrap required, opened a new log
I20260929 19:14:31.888384 32500 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2c8e149cc71401c88376ba1e0f4bc7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5110fcca9ce049c48662467052181669" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36913 } } peers { permanent_uuid: "c2c8e149cc71401c88376ba1e0f4bc7b" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } } peers { permanent_uuid: "c57b0f9c5f474f2cb2d8985ad762fc73" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } }
I20260929 19:14:31.888530 32500 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2c8e149cc71401c88376ba1e0f4bc7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2c8e149cc71401c88376ba1e0f4bc7b, State: Initialized, Role: FOLLOWER
I20260929 19:14:31.888623 32500 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2c8e149cc71401c88376ba1e0f4bc7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5110fcca9ce049c48662467052181669" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36913 } } peers { permanent_uuid: "c2c8e149cc71401c88376ba1e0f4bc7b" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } } peers { permanent_uuid: "c57b0f9c5f474f2cb2d8985ad762fc73" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } }
I20260929 19:14:31.888945 32500 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2c8e149cc71401c88376ba1e0f4bc7b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:31.889159 32473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2c8e149cc71401c88376ba1e0f4bc7b [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:31.890262 32473 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c2c8e149cc71401c88376ba1e0f4bc7b [term 1 FOLLOWER]: Refusing update from remote peer 5110fcca9ce049c48662467052181669: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:14:31.890800 32608 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2c8e149cc71401c88376ba1e0f4bc7b" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:14:31.890504 32474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5110fcca9ce049c48662467052181669" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c2c8e149cc71401c88376ba1e0f4bc7b"
I20260929 19:14:31.891994 32474 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c2c8e149cc71401c88376ba1e0f4bc7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5110fcca9ce049c48662467052181669 in term 1.
W20260929 19:14:31.894891 32635 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c2c8e149cc71401c88376ba1e0f4bc7b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:14:31.895071 32635 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:14:31.895215 32624 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2c8e149cc71401c88376ba1e0f4bc7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5110fcca9ce049c48662467052181669" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5110fcca9ce049c48662467052181669" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36913 } } peers { permanent_uuid: "c2c8e149cc71401c88376ba1e0f4bc7b" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } } peers { permanent_uuid: "c57b0f9c5f474f2cb2d8985ad762fc73" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } } }
I20260929 19:14:31.895298 32624 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2c8e149cc71401c88376ba1e0f4bc7b [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:14:31.895341 32624 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2c8e149cc71401c88376ba1e0f4bc7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5110fcca9ce049c48662467052181669. Latest consensus state: current_term: 1 leader_uuid: "5110fcca9ce049c48662467052181669" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5110fcca9ce049c48662467052181669" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36913 } } peers { permanent_uuid: "c2c8e149cc71401c88376ba1e0f4bc7b" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } } peers { permanent_uuid: "c57b0f9c5f474f2cb2d8985ad762fc73" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } } }
I20260929 19:14:31.895390 32624 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2c8e149cc71401c88376ba1e0f4bc7b [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:14:31.895414 32624 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2c8e149cc71401c88376ba1e0f4bc7b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5110fcca9ce049c48662467052181669" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5110fcca9ce049c48662467052181669" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36913 } } peers { permanent_uuid: "c2c8e149cc71401c88376ba1e0f4bc7b" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } } peers { permanent_uuid: "c57b0f9c5f474f2cb2d8985ad762fc73" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } } }
I20260929 19:14:31.895470 32624 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2c8e149cc71401c88376ba1e0f4bc7b [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:14:31.897198 32612 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669: Generated new TSK 0
I20260929 19:14:31.897493 32612 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:31.897673 32637 mvcc.cc:205] Tried to move back new op lower bound from 7334745177615380480 to 7334745177443213312. Current Snapshot: MvccSnapshot[applied={T|T < 7334745177504276480}]
I20260929 19:14:31.899262 32595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5110fcca9ce049c48662467052181669" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5110fcca9ce049c48662467052181669" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 36913 } } peers { permanent_uuid: "c2c8e149cc71401c88376ba1e0f4bc7b" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 36089 } } peers { permanent_uuid: "c57b0f9c5f474f2cb2d8985ad762fc73" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 36091 } } }
I20260929 19:14:31.899372 32595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5110fcca9ce049c48662467052181669 [sys.catalog]: This master's current role is: LEADER
W20260929 19:14:31.917745 32505 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:31.915964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34308 (local address 127.9.236.61:36089)
0929 19:14:31.916029 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.916031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.916961 (+ 930us) server_negotiation.cc:407] Connection header received
0929 19:14:31.917091 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.917093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.917125 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.917161 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.917655 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34308: BlockingRecv error: recv got EOF from 127.0.0.1:34308 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260929 19:14:31.918766 32579 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:31.916364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37204 (local address 127.9.236.60:36091)
0929 19:14:31.916414 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.916416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.918273 (+ 1857us) server_negotiation.cc:407] Connection header received
0929 19:14:31.918371 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.918372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.918384 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.918410 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.918700 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37204: BlockingRecv error: recv got EOF from 127.0.0.1:37204 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260929 19:14:31.919785 32428 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:31.916690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55494 (local address 127.9.236.62:36913)
0929 19:14:31.916738 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.916740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.919198 (+ 2458us) server_negotiation.cc:407] Connection header received
0929 19:14:31.919308 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.919309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.919322 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.919350 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.919711 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55494: BlockingRecv error: recv got EOF from 127.0.0.1:55494 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260929 19:14:31.922550 32505 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:31.921789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34316 (local address 127.9.236.61:36089)
0929 19:14:31.921825 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.921826 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.922046 (+ 220us) server_negotiation.cc:407] Connection header received
0929 19:14:31.922152 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.922153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.922175 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.922203 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.922490 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34316: BlockingRecv error: recv got EOF from 127.0.0.1:34316 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260929 19:14:31.925647 32505 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:31.924898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34326 (local address 127.9.236.61:36089)
0929 19:14:31.924929 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.924931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.925202 (+ 271us) server_negotiation.cc:407] Connection header received
0929 19:14:31.925288 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.925289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.925298 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.925325 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.925607 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34326: BlockingRecv error: recv got EOF from 127.0.0.1:34326 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260929 19:14:31.929646 32505 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:31.928829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34340 (local address 127.9.236.61:36089)
0929 19:14:31.928877 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.928878 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.929192 (+ 314us) server_negotiation.cc:407] Connection header received
0929 19:14:31.929285 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.929285 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.929295 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.929322 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.929602 (+ 280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34340: BlockingRecv error: recv got EOF from 127.0.0.1:34340 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260929 19:14:31.935531 32505 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:31.934787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34356 (local address 127.9.236.61:36089)
0929 19:14:31.934823 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.934825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.935068 (+ 243us) server_negotiation.cc:407] Connection header received
0929 19:14:31.935160 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.935161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.935170 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.935197 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.935487 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34356: BlockingRecv error: recv got EOF from 127.0.0.1:34356 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260929 19:14:31.945991 32505 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:31.944941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34360 (local address 127.9.236.61:36089)
0929 19:14:31.945015 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.945018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.945354 (+ 336us) server_negotiation.cc:407] Connection header received
0929 19:14:31.945461 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.945463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.945475 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.945505 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.945910 (+ 405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34360: BlockingRecv error: recv got EOF from 127.0.0.1:34360 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260929 19:14:31.965018 32505 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:31.963823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34376 (local address 127.9.236.61:36089)
0929 19:14:31.963881 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.963884 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.964227 (+ 343us) server_negotiation.cc:407] Connection header received
0929 19:14:31.964343 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.964344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.964358 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.964391 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.964849 (+ 458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34376: BlockingRecv error: recv got EOF from 127.0.0.1:34376 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260929 19:14:32.000051 32505 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:31.999016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34386 (local address 127.9.236.61:36089)
0929 19:14:31.999071 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:31.999074 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:31.999384 (+ 310us) server_negotiation.cc:407] Connection header received
0929 19:14:31.999490 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:31.999491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:31.999508 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:31.999538 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:31.999973 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34386: BlockingRecv error: recv got EOF from 127.0.0.1:34386 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260929 19:14:32.066687 32505 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:32.065894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34390 (local address 127.9.236.61:36089)
0929 19:14:32.065975 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:32.065977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:32.066029 (+ 52us) server_negotiation.cc:407] Connection header received
0929 19:14:32.066126 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:32.066127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:32.066138 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:32.066205 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:32.066626 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34390: BlockingRecv error: recv got EOF from 127.0.0.1:34390 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260929 19:14:32.196894 32505 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:32.196115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34404 (local address 127.9.236.61:36089)
0929 19:14:32.196240 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:32.196242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:32.196300 (+ 58us) server_negotiation.cc:407] Connection header received
0929 19:14:32.196327 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:32.196329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:32.196339 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:32.196389 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:32.196832 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34404: BlockingRecv error: recv got EOF from 127.0.0.1:34404 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260929 19:14:32.454938 32505 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:32.454354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34412 (local address 127.9.236.61:36089)
0929 19:14:32.454439 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:32.454441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:32.454450 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:14:32.454474 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:32.454475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:32.454487 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:32.454532 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:32.454880 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34412: BlockingRecv error: recv got EOF from 127.0.0.1:34412 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 19:14:32.822646 32592 catalog_manager.cc:1259] Loaded cluster ID: be4e24a03c644d8daa33bff1d81e5865
I20260929 19:14:32.822755 32592 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c57b0f9c5f474f2cb2d8985ad762fc73: loading cluster ID for follower catalog manager: success
I20260929 19:14:32.823657 32592 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c57b0f9c5f474f2cb2d8985ad762fc73: acquiring CA information for follower catalog manager: success
I20260929 19:14:32.824007 32592 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c57b0f9c5f474f2cb2d8985ad762fc73: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 19:14:32.895560 32635 catalog_manager.cc:1259] Loaded cluster ID: be4e24a03c644d8daa33bff1d81e5865
I20260929 19:14:32.895637 32635 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c2c8e149cc71401c88376ba1e0f4bc7b: loading cluster ID for follower catalog manager: success
I20260929 19:14:32.896457 32635 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c2c8e149cc71401c88376ba1e0f4bc7b: acquiring CA information for follower catalog manager: success
I20260929 19:14:32.896747 32635 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c2c8e149cc71401c88376ba1e0f4bc7b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 19:14:32.970904 32708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:32.968219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34426 (local address 127.9.236.61:36089)
0929 19:14:32.968367 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:32.968369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:32.968380 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:14:32.968416 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:32.968417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:32.968482 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:32.968602 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:32.968771 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:32.969374 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:32.970164 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:32.970369 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:32.970417 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:32.970585 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:32.970710 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:32.970776 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:32.970800 (+ 24us) server_negotiation.cc:299] Negotiation successful
0929 19:14:32.970823 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260929 19:14:32.975306 32714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:32.972867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37214 (local address 127.9.236.60:36091)
0929 19:14:32.973009 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:32.973011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:32.973069 (+ 58us) server_negotiation.cc:407] Connection header received
0929 19:14:32.973202 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:32.973203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:32.973244 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:32.973317 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:32.973417 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:32.973904 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:32.974593 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:32.974806 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:32.974859 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:32.974994 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:32.975103 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:32.975170 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:32.975197 (+ 27us) server_negotiation.cc:299] Negotiation successful
0929 19:14:32.975218 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260929 19:14:32.979703 32708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:32.977058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34430 (local address 127.9.236.61:36089)
0929 19:14:32.977210 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:32.977213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:32.977229 (+ 16us) server_negotiation.cc:407] Connection header received
0929 19:14:32.977336 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:32.977338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:32.977351 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:32.977397 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:32.977550 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:32.978248 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:32.979095 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:32.979289 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:32.979340 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:32.979472 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:32.979505 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:32.979543 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:32.979598 (+ 55us) server_negotiation.cc:299] Negotiation successful
0929 19:14:32.979618 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260929 19:14:32.980607 32714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:32.977410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37226 (local address 127.9.236.60:36091)
0929 19:14:32.977449 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:32.977451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:32.977721 (+ 270us) server_negotiation.cc:407] Connection header received
0929 19:14:32.977812 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:32.977813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:32.977828 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:32.977881 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:32.978000 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:32.978532 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:32.979743 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:32.980038 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:32.980081 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:32.980306 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:32.980346 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:32.980411 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:32.980508 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 19:14:32.980540 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:14:32.980873 32721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:32.977072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55504 (local address 127.9.236.62:36913)
0929 19:14:32.977300 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:32.977314 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:32.977327 (+ 13us) server_negotiation.cc:407] Connection header received
0929 19:14:32.977384 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:32.977385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:32.977457 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:32.977564 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:32.977760 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:32.978495 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:32.979744 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:32.980080 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:32.980130 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:32.980402 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:32.980526 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:32.980592 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:32.980752 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 19:14:32.980791 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":79,"threads_started":1}
I20260929 19:14:32.982792 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 32357
I20260929 19:14:32.987583 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 32432
I20260929 19:14:32.993175 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 32507
2026-09-29T19:14:32Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2735 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 19:14:33 dist-test-slave-wftq krb5kdc[32725](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 19:14:33 dist-test-slave-wftq krb5kdc[32725](info): set up 2 sockets
Sep 29 19:14:33 dist-test-slave-wftq krb5kdc[32725](info): commencing operation
krb5kdc: starting...
W20260929 19:14:33.949443 10160 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/test-user.keytab.
Sep 29 19:14:34 dist-test-slave-wftq krb5kdc[32725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T19:14:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T19:14:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.236.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
Entry for principal kudu/127.9.236.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab.
I20260929 19:14:34.075629 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:38803
--webserver_interface=127.9.236.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:44653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:38803,127.9.236.61:41435,127.9.236.60:38141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 19:14:34.154917 32738 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 19:14:34.155146 32738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:34.155162 32738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:34.155182 32738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:34.156315 32738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:34.156359 32738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:34.156373 32738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:34.156383 32738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:34.156394 32738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:34.157963 32738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:44653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:38803,127.9.236.61:41435,127.9.236.60:38141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.62.keytab
--principal=kudu/127.9.236.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.62:38803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.236.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:34.158177 32738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:34.158372 32738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:34.163486 32746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:34.163560 32743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:34.163560 32744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:34.163810 32738 server_base.cc:1034] running on GCE node
I20260929 19:14:34.164077 32738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:34.164354 32738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:34.165488 32738 hybrid_clock.cc:648] HybridClock initialized: now 1790709274165466 us; error 44 us; skew 500 ppm
Sep 29 19:14:34 dist-test-slave-wftq krb5kdc[32725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709274, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:34.167975 32738 init.cc:377] Logged in from keytab as kudu/127.9.236.62@KRBTEST.COM (short username kudu)
I20260929 19:14:34.168419 32738 webserver.cc:460] Webserver started at http://127.9.236.62:37193/ using document root <none> and password file <none>
I20260929 19:14:34.168638 32738 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:34.168694 32738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:34.168826 32738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:34.169907 32738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-0/data/instance:
uuid: "f08803b1193542f5855a0f23f7fd086c"
format_stamp: "Formatted at 2026-09-29 19:14:34 on dist-test-slave-wftq"
server_key: "3d094b90f59f9d2aa152c250d8091897"
server_key_iv: "1ff6478b2948bee428c8fb3155688a5d"
server_key_version: "encryptionkey@0"
I20260929 19:14:34.170228 32738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-0/wal/instance:
uuid: "f08803b1193542f5855a0f23f7fd086c"
format_stamp: "Formatted at 2026-09-29 19:14:34 on dist-test-slave-wftq"
server_key: "3d094b90f59f9d2aa152c250d8091897"
server_key_iv: "1ff6478b2948bee428c8fb3155688a5d"
server_key_version: "encryptionkey@0"
I20260929 19:14:34.171505 32738 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:14:34.172324 32753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:34.172509 32738 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:34.172595 32738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-0/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-0/wal
uuid: "f08803b1193542f5855a0f23f7fd086c"
format_stamp: "Formatted at 2026-09-29 19:14:34 on dist-test-slave-wftq"
server_key: "3d094b90f59f9d2aa152c250d8091897"
server_key_iv: "1ff6478b2948bee428c8fb3155688a5d"
server_key_version: "encryptionkey@0"
I20260929 19:14:34.172667 32738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:34.192642 32738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:34.194540 32738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:34.194660 32738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:34.199018 32738 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.62:38803
I20260929 19:14:34.199101 337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.62:38803 every 8 connection(s)
I20260929 19:14:34.199431 32738 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-0/data/info.pb
I20260929 19:14:34.200589 338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:34.201211 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 32738
I20260929 19:14:34.201330 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-0/wal/instance
I20260929 19:14:34.201537 10160 external_mini_cluster.cc:1428] Setting key 172361badfb5b7008b78e87af22332bd
I20260929 19:14:34.203079 338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38803 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.236.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.61@KRBTEST.COM" created.
Sep 29 19:14:34 dist-test-slave-wftq krb5kdc[32725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709274, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:14:34.216665 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.204055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38803 (local address 127.0.0.1:45764)
0929 19:14:34.204299 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.204344 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:34.204936 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:34.205243 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:34.205246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:34.205568 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:34.205784 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:34.205789 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.206833 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.206834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:34.207556 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:34.207558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.208046 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.208047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:34.208110 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.208690 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:34.208700 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:34.212822 (+ 4122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:34.214976 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:34.214985 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:34.214989 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:34.215215 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:34.215514 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:34.215514 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:34.215515 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:34.215603 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:34.216054 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:34.216059 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:34.216129 (+ 70us) client_negotiation.cc:769] Sending connection context
0929 19:14:34.216210 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 19:14:34.216377 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":148,"threads_started":1}
I20260929 19:14:34.216668 341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.204075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45764 (local address 127.9.236.62:38803)
0929 19:14:34.204287 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.204298 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.204476 (+ 178us) server_negotiation.cc:407] Connection header received
0929 19:14:34.205094 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.205096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.205147 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.205233 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.205876 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.206742 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.207635 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.207953 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.208065 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.212955 (+ 4890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:34.212965 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:34.212966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:34.212975 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:34.214874 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.215313 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.215316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.215317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.215441 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.215678 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.215679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.215679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.215985 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:34.216078 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.216272 (+ 194us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.216469 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":162,"thread_start_us":117,"threads_started":1}
I20260929 19:14:34.217655 338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:14:34.218043 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.217931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:41435 (local address 127.0.0.1:43486)
0929 19:14:34.217979 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.218024 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.61:41435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 19:14:34.218183 32755 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.236.61:41435: connect: Connection refused (error 111)
I20260929 19:14:34.218691 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.218602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:41435 (local address 127.0.0.1:43490)
0929 19:14:34.218646 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.218670 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.61:41435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 19:14:34.218791 338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.61:41435: Network error: Client connection negotiation failed: client connection to 127.9.236.61:41435: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.61.keytab.
Entry for principal kudu/127.9.236.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.61.keytab.
I20260929 19:14:34.228955 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.61:41435
--webserver_interface=127.9.236.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.61.keytab
--principal=kudu/127.9.236.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:44653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:38803,127.9.236.61:41435,127.9.236.60:38141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:14:34.268050 338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } attempt: 1
I20260929 19:14:34.268668 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.268523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:41435 (local address 127.0.0.1:43498)
0929 19:14:34.268587 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.268635 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.61:41435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260929 19:14:34.269071 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.268987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:41435 (local address 127.0.0.1:43512)
0929 19:14:34.269029 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.269054 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.61:41435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260929 19:14:34.269182 338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.61:41435: Network error: Client connection negotiation failed: client connection to 127.9.236.61:41435: connect: Connection refused (error 111)
W20260929 19:14:34.308987 345 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 19:14:34.309242 345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:34.309266 345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:34.309290 345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:34.310463 345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:34.310521 345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:34.310540 345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:34.310555 345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:34.310570 345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:34.312098 345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:44653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:38803,127.9.236.61:41435,127.9.236.60:38141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.61.keytab
--principal=kudu/127.9.236.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.61:41435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.236.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:34.312340 345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:34.312552 345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:34.317682 350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:34.317700 353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:34.317750 351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:34.317921 345 server_base.cc:1034] running on GCE node
I20260929 19:14:34.318162 345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:34.318437 345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:34.319574 345 hybrid_clock.cc:648] HybridClock initialized: now 1790709274319552 us; error 50 us; skew 500 ppm
Sep 29 19:14:34 dist-test-slave-wftq krb5kdc[32725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709274, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:34.322203 345 init.cc:377] Logged in from keytab as kudu/127.9.236.61@KRBTEST.COM (short username kudu)
I20260929 19:14:34.322655 345 webserver.cc:460] Webserver started at http://127.9.236.61:42195/ using document root <none> and password file <none>
I20260929 19:14:34.322914 345 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:34.322971 345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:34.323105 345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:34.324159 345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-1/data/instance:
uuid: "a21497ce397e4b27b594e197f24736a0"
format_stamp: "Formatted at 2026-09-29 19:14:34 on dist-test-slave-wftq"
server_key: "0151c35619b1b81447f1a65c6e8b303b"
server_key_iv: "f537001b9a2d86056f1f7a3afb20b905"
server_key_version: "encryptionkey@0"
I20260929 19:14:34.324476 345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-1/wal/instance:
uuid: "a21497ce397e4b27b594e197f24736a0"
format_stamp: "Formatted at 2026-09-29 19:14:34 on dist-test-slave-wftq"
server_key: "0151c35619b1b81447f1a65c6e8b303b"
server_key_iv: "f537001b9a2d86056f1f7a3afb20b905"
server_key_version: "encryptionkey@0"
I20260929 19:14:34.325834 345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260929 19:14:34.326640 360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:34.326841 345 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 19:14:34.326924 345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-1/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-1/wal
uuid: "a21497ce397e4b27b594e197f24736a0"
format_stamp: "Formatted at 2026-09-29 19:14:34 on dist-test-slave-wftq"
server_key: "0151c35619b1b81447f1a65c6e8b303b"
server_key_iv: "f537001b9a2d86056f1f7a3afb20b905"
server_key_version: "encryptionkey@0"
I20260929 19:14:34.326985 345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:34.337347 338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } attempt: 2
I20260929 19:14:34.337956 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.337814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:41435 (local address 127.0.0.1:43524)
0929 19:14:34.337879 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.337921 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.61:41435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260929 19:14:34.338346 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.338251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:41435 (local address 127.0.0.1:43538)
0929 19:14:34.338295 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.338325 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.61:41435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260929 19:14:34.338441 338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.61:41435: Network error: Client connection negotiation failed: client connection to 127.9.236.61:41435: connect: Connection refused (error 111)
I20260929 19:14:34.348672 345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:34.350617 345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:34.350760 345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:34.355010 345 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.61:41435
I20260929 19:14:34.355082 412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.61:41435 every 8 connection(s)
I20260929 19:14:34.355407 345 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-1/data/info.pb
I20260929 19:14:34.356508 413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:34.358565 413 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38803 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 19:14:34 dist-test-slave-wftq krb5kdc[32725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709274, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.61@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:14:34.364663 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 345
I20260929 19:14:34.364785 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-1/wal/instance
I20260929 19:14:34.364981 10160 external_mini_cluster.cc:1428] Setting key 2b7be97c339b923e6ddb8c7644a11a11
I20260929 19:14:34.368223 341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.359467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45774 (local address 127.9.236.62:38803)
0929 19:14:34.359532 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.359534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.359699 (+ 165us) server_negotiation.cc:407] Connection header received
0929 19:14:34.360321 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.360322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.360342 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.360390 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.361017 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.361575 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.362521 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.362736 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.362793 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.365225 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:34.365227 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:34.365228 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:34.365238 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:34.366854 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.367267 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.367268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.367269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.367334 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.367582 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.367582 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.367583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.367714 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:34.367767 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.368070 (+ 303us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.368147 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 19:14:34.368417 415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.359384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38803 (local address 127.0.0.1:45774)
0929 19:14:34.359563 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.359607 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:34.360216 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:34.360457 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:34.360461 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:34.360718 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:34.360926 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:34.360930 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.361658 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.361659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:34.362441 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:34.362444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.362819 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.362820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:34.362934 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.363473 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:34.363480 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:34.365084 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:34.366958 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:34.366964 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:34.366966 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:34.367185 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:34.367397 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:34.367398 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:34.367398 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:34.367516 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:34.367776 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:34.367781 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:34.367936 (+ 155us) client_negotiation.cc:769] Sending connection context
0929 19:14:34.368051 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 19:14:34.368204 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":79,"threads_started":1}
I20260929 19:14:34.369127 413 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.236.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.236.60@KRBTEST.COM" created.
Sep 29 19:14:34 dist-test-slave-wftq krb5kdc[32725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709274, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.61@KRBTEST.COM for kudu/127.9.236.61@KRBTEST.COM
I20260929 19:14:34.379047 415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.369433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:41435 (local address 127.0.0.1:43540)
0929 19:14:34.369480 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.369491 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:34.369529 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:34.369848 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:34.369848 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:34.369999 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:34.370055 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:34.370056 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.370915 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.370916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:34.371430 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:34.371432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.371889 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.371889 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:34.371960 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.372418 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:34.372423 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:34.375859 (+ 3436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:34.377855 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:34.377856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:34.377857 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:34.378040 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:34.378352 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:34.378353 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:34.378354 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:34.378413 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:34.378694 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:34.378695 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:34.378734 (+ 39us) client_negotiation.cc:769] Sending connection context
0929 19:14:34.378795 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 19:14:34.378922 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260929 19:14:34.379067 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.369534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43540 (local address 127.9.236.61:41435)
0929 19:14:34.369711 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.369716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.369738 (+ 22us) server_negotiation.cc:407] Connection header received
0929 19:14:34.369756 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.369757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.369769 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.369838 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.370116 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.370837 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.371495 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.371829 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.371887 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.375978 (+ 4091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:34.375992 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:34.375992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:34.376001 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:34.377740 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.378145 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.378148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.378149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.378211 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.378467 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.378468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.378468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.378636 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:34.378712 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.378824 (+ 112us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.378972 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":143,"thread_start_us":93,"threads_started":1}
I20260929 19:14:34.379961 413 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:14:34.380386 415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.380208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:38141 (local address 127.0.0.1:38676)
0929 19:14:34.380275 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.380341 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 19:14:34.380558 361 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.236.60:38141: connect: Connection refused (error 111)
I20260929 19:14:34.381022 415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.380910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:38141 (local address 127.0.0.1:38682)
0929 19:14:34.380967 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.381000 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 19:14:34.381165 413 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.60:38141: Network error: Client connection negotiation failed: client connection to 127.9.236.60:38141: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.236.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.60.keytab.
Entry for principal kudu/127.9.236.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.60.keytab.
I20260929 19:14:34.392511 10160 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0UA8iT/build/release/bin/kudu
/tmp/dist-test-task0UA8iT/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.60:38141
--webserver_interface=127.9.236.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.60.keytab
--principal=kudu/127.9.236.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.236.20:44653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.236.62:38803,127.9.236.61:41435,127.9.236.60:38141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 19:14:34.429596 338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } attempt: 3
I20260929 19:14:34.430377 413 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } attempt: 1
I20260929 19:14:34.430850 415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.430637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:38141 (local address 127.0.0.1:38688)
0929 19:14:34.430709 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.430779 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260929 19:14:34.431345 415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.431197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:38141 (local address 127.0.0.1:38690)
0929 19:14:34.431263 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.431308 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260929 19:14:34.431463 413 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.60:38141: Network error: Client connection negotiation failed: client connection to 127.9.236.60:38141: connect: Connection refused (error 111)
Sep 29 19:14:34 dist-test-slave-wftq krb5kdc[32725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709274, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for kudu/127.9.236.61@KRBTEST.COM
I20260929 19:14:34.438156 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.430108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:41435 (local address 127.0.0.1:43554)
0929 19:14:34.430221 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.430236 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:34.430351 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:34.430659 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:34.430662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:34.430825 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:34.430899 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:34.430901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.431749 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.431750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:34.432674 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:34.432676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.433077 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.433078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:34.433195 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.433698 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:34.433704 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:34.435334 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:34.437066 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:34.437067 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:34.437068 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:34.437295 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:34.437500 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:34.437501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:34.437501 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:34.437562 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:34.437859 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:34.437860 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:34.437915 (+ 55us) client_negotiation.cc:769] Sending connection context
0929 19:14:34.437980 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 19:14:34.438084 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260929 19:14:34.438210 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.430157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43554 (local address 127.9.236.61:41435)
0929 19:14:34.430212 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.430214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.430305 (+ 91us) server_negotiation.cc:407] Connection header received
0929 19:14:34.430431 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.430433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.430447 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.430583 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.430981 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.431653 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.432769 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.433001 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.433064 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.435437 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:34.435439 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:34.435440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:34.435449 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:34.436951 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.437372 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.437373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.437373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.437437 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.437621 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.437621 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.437622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.437771 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:34.437838 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.438018 (+ 180us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.438108 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260929 19:14:34.438742 338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:14:34.439100 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.438965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:38141 (local address 127.0.0.1:38698)
0929 19:14:34.439019 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.439059 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260929 19:14:34.439494 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.439401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:38141 (local address 127.0.0.1:38710)
0929 19:14:34.439450 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.439477 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 19:14:34.439652 338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.60:38141: Network error: Client connection negotiation failed: client connection to 127.9.236.60:38141: connect: Connection refused (error 111)
I20260929 19:14:34.470829 338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } attempt: 1
I20260929 19:14:34.471390 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.471153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:38141 (local address 127.0.0.1:38716)
0929 19:14:34.471239 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.471334 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260929 19:14:34.471863 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.471739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:38141 (local address 127.0.0.1:38722)
0929 19:14:34.471789 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.471819 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 19:14:34.471966 338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.60:38141: Network error: Client connection negotiation failed: client connection to 127.9.236.60:38141: connect: Connection refused (error 111)
W20260929 19:14:34.483011 420 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 19:14:34.483246 420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 19:14:34.483263 420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 19:14:34.483282 420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 19:14:34.484472 420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 19:14:34.484515 420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 19:14:34.484529 420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 19:14:34.484539 420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 19:14:34.484550 420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 19:14:34.486099 420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.236.20:44653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.236.62:38803,127.9.236.61:41435,127.9.236.60:38141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/krb5kdc/kudu_127.9.236.60.keytab
--principal=kudu/127.9.236.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.236.60:38141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.236.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 29 Sep 2026 18:09:13 UTC on ccycloud.quasar-hpkalf.root.comops.site
build id 4680913
I20260929 19:14:34.486351 420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 19:14:34.486579 420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 19:14:34.491808 428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:34.491954 420 server_base.cc:1034] running on GCE node
W20260929 19:14:34.491815 425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 19:14:34.491847 426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 19:14:34.492383 420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 19:14:34.492703 420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 19:14:34.493868 420 hybrid_clock.cc:648] HybridClock initialized: now 1790709274493846 us; error 41 us; skew 500 ppm
Sep 29 19:14:34 dist-test-slave-wftq krb5kdc[32725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709274, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 19:14:34.496335 420 init.cc:377] Logged in from keytab as kudu/127.9.236.60@KRBTEST.COM (short username kudu)
I20260929 19:14:34.496753 420 webserver.cc:460] Webserver started at http://127.9.236.60:45389/ using document root <none> and password file <none>
I20260929 19:14:34.496938 420 fs_manager.cc:359] Metadata directory not provided
I20260929 19:14:34.496980 420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 19:14:34.497085 420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 19:14:34.498186 420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-2/data/instance:
uuid: "e9b029c03701466f9cc8d5650dd9c0e4"
format_stamp: "Formatted at 2026-09-29 19:14:34 on dist-test-slave-wftq"
server_key: "7dad1deb5afe8d0c5204529ff8b46255"
server_key_iv: "3a99e6f0cd772eb41f0a20b74c48a88c"
server_key_version: "encryptionkey@0"
I20260929 19:14:34.498492 420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-2/wal/instance:
uuid: "e9b029c03701466f9cc8d5650dd9c0e4"
format_stamp: "Formatted at 2026-09-29 19:14:34 on dist-test-slave-wftq"
server_key: "7dad1deb5afe8d0c5204529ff8b46255"
server_key_iv: "3a99e6f0cd772eb41f0a20b74c48a88c"
server_key_version: "encryptionkey@0"
I20260929 19:14:34.499594 413 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } attempt: 2
I20260929 19:14:34.499776 420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 19:14:34.500069 415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.499917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:38141 (local address 127.0.0.1:38732)
0929 19:14:34.499991 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.500035 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260929 19:14:34.500484 415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.500394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:38141 (local address 127.0.0.1:38738)
0929 19:14:34.500436 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.500463 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.236.60:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260929 19:14:34.500617 413 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.236.60:38141: Network error: Client connection negotiation failed: client connection to 127.9.236.60:38141: connect: Connection refused (error 111)
I20260929 19:14:34.500682 435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 19:14:34.500883 420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 19:14:34.500968 420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-2/data,/tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-2/wal
uuid: "e9b029c03701466f9cc8d5650dd9c0e4"
format_stamp: "Formatted at 2026-09-29 19:14:34 on dist-test-slave-wftq"
server_key: "7dad1deb5afe8d0c5204529ff8b46255"
server_key_iv: "3a99e6f0cd772eb41f0a20b74c48a88c"
server_key_version: "encryptionkey@0"
I20260929 19:14:34.501051 420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 19:14:34.513080 420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 19:14:34.515077 420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 19:14:34.515195 420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 19:14:34.519227 420 rpc_server.cc:307] RPC server started. Bound to: 127.9.236.60:38141
I20260929 19:14:34.519295 487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.236.60:38141 every 8 connection(s)
I20260929 19:14:34.519620 420 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-2/data/info.pb
I20260929 19:14:34.520706 488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 19:14:34.522560 488 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38803 } has no permanent_uuid. Determining permanent_uuid...
I20260929 19:14:34.528074 10160 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0UA8iT/build/release/bin/kudu as pid 420
I20260929 19:14:34.528227 10160 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0UA8iT/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790709008521303-10160-0/minicluster-data/master-2/wal/instance
I20260929 19:14:34.528456 10160 external_mini_cluster.cc:1428] Setting key 578737c170d4a726782e78b5d29e487f
Sep 29 19:14:34 dist-test-slave-wftq krb5kdc[32725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709274, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.60@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
I20260929 19:14:34.532294 341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.523378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45780 (local address 127.9.236.62:38803)
0929 19:14:34.523446 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.523448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.523681 (+ 233us) server_negotiation.cc:407] Connection header received
0929 19:14:34.524194 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.524195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.524206 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.524278 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.524958 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.525477 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.526526 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.526759 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.526823 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.529462 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:34.529464 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:34.529465 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:34.529473 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:34.530902 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.531324 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.531325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.531326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.531387 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.531646 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.531646 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.531647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.531784 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:34.531860 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.532149 (+ 289us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.532218 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 19:14:34.532586 490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.523325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.62:38803 (local address 127.0.0.1:45780)
0929 19:14:34.523539 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.523585 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:34.524096 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:34.524351 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:34.524354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:34.524634 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:34.524884 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:34.524888 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.525548 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.525548 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:34.526428 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:34.526429 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.526887 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.526888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:34.527022 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.527507 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:34.527516 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:34.529314 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:34.530990 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:34.530996 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:34.530999 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:34.531253 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:34.531454 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:34.531454 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:34.531455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:34.531587 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:34.531850 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:34.531855 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:34.532013 (+ 158us) client_negotiation.cc:769] Sending connection context
0929 19:14:34.532130 (+ 117us) client_negotiation.cc:240] Negotiation successful
0929 19:14:34.532301 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":111,"threads_started":1}
I20260929 19:14:34.533234 488 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 19:14:34 dist-test-slave-wftq krb5kdc[32725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.60@KRBTEST.COM
Sep 29 19:14:34 dist-test-slave-wftq krb5kdc[32725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709274, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.60@KRBTEST.COM for kudu/127.9.236.61@KRBTEST.COM
I20260929 19:14:34.537755 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.528804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38746 (local address 127.9.236.60:38141)
0929 19:14:34.529019 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.529024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.529138 (+ 114us) server_negotiation.cc:407] Connection header received
0929 19:14:34.529203 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.529205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.529232 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.529349 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.530098 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.530905 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.531833 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.532275 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.532614 (+ 339us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.535041 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:34.535050 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:34.535050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:34.535057 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:34.536607 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.536935 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.536938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.536939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.536999 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.537182 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.537183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.537183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.537330 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:34.537481 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.537577 (+ 96us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.537653 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":105,"threads_started":1}
I20260929 19:14:34.540565 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.533661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43560 (local address 127.9.236.61:41435)
0929 19:14:34.533718 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.533720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.533729 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:14:34.533758 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.533759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.533770 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.533841 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.534101 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.534596 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.535401 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.535601 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.535658 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.537656 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:34.537658 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:34.537659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:34.537666 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:34.539353 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.539680 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.539681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.539682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.539744 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.539940 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.539941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.539941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.540086 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:34.540139 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.540356 (+ 217us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.540444 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260929 19:14:34.540908 490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.533545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.61:41435 (local address 127.0.0.1:43560)
0929 19:14:34.533609 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.533622 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:34.533666 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:34.533841 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:34.533842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:34.533961 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:34.534010 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:34.534012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.534674 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.534676 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:34.535317 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:34.535319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.535672 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.535673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:34.535764 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.536187 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:34.536192 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:34.537572 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:34.539444 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:34.539445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:34.539446 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:34.539609 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:34.539806 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:34.539806 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:34.539807 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:34.539886 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:34.540148 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:34.540148 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:34.540194 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 19:14:34.540663 (+ 469us) client_negotiation.cc:240] Negotiation successful
0929 19:14:34.540770 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 19:14:34.541363 488 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 19:14:34 dist-test-slave-wftq krb5kdc[32725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.62@KRBTEST.COM
Sep 29 19:14:34 dist-test-slave-wftq krb5kdc[32725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709274, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.60@KRBTEST.COM for kudu/127.9.236.60@KRBTEST.COM
I20260929 19:14:34.546350 341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.538693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45786 (local address 127.9.236.62:38803)
0929 19:14:34.538756 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.538758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.538804 (+ 46us) server_negotiation.cc:407] Connection header received
0929 19:14:34.538880 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.538881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.538892 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.538944 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.539404 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.539982 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.540857 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.541152 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.541220 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.543514 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:34.543516 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:34.543516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:34.543524 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:34.545168 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.545571 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.545572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.545573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.545628 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.545832 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.545833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.545833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.545952 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:34.546078 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.546179 (+ 101us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.546256 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 19:14:34.547116 338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } attempt: 2
I20260929 19:14:34.549276 490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.541671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:38141 (local address 127.0.0.1:38754)
0929 19:14:34.541707 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.541717 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:34.541754 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:34.541928 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:34.541929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:34.542062 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:34.542097 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:34.542098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.542832 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.542833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:34.543398 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:34.543400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.544154 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.544156 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:34.544225 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.544593 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:34.544599 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:34.546110 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:34.548093 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:34.548095 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:34.548095 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:34.548302 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:34.548525 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:34.548525 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:34.548526 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:34.548587 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:34.548856 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:34.548857 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:34.548893 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 19:14:34.548957 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 19:14:34.549056 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260929 19:14:34.549377 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.541789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38754 (local address 127.9.236.60:38141)
0929 19:14:34.541846 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.541848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.541854 (+ 6us) server_negotiation.cc:407] Connection header received
0929 19:14:34.541870 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.541870 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.541879 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.541920 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.542152 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.542721 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.543742 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.544056 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.544114 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.546202 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:34.546203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:34.546204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:34.546211 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:34.547763 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.548385 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.548386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.548386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.548448 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.548652 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.548652 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.548653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.548772 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:34.548833 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.549018 (+ 185us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.549141 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Sep 29 19:14:34 dist-test-slave-wftq krb5kdc[32725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709274, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.62@KRBTEST.COM for kudu/127.9.236.60@KRBTEST.COM
Sep 29 19:14:34 dist-test-slave-wftq krb5kdc[32725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.236.61@KRBTEST.COM
I20260929 19:14:34.552362 488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9b029c03701466f9cc8d5650dd9c0e4: Bootstrap starting.
I20260929 19:14:34.553542 488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9b029c03701466f9cc8d5650dd9c0e4: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:34.553839 488 log.cc:824] T 00000000000000000000000000000000 P e9b029c03701466f9cc8d5650dd9c0e4: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:34.554777 488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9b029c03701466f9cc8d5650dd9c0e4: No bootstrap required, opened a new log
I20260929 19:14:34.555395 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.547343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:38141 (local address 127.0.0.1:38762)
0929 19:14:34.547453 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.547462 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:34.547494 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:34.547841 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:34.547842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:34.547978 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:34.548013 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:34.548015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.548869 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.548871 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:34.549651 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:34.549654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.550163 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.550164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:34.550224 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.550635 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:34.550640 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:34.551773 (+ 1133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:34.553520 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:34.553521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:34.553522 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:34.553755 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:34.554711 (+ 956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:34.554712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:34.554713 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:34.554785 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:34.555067 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:34.555068 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:34.555115 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 19:14:34.555177 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 19:14:34.555283 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260929 19:14:34.555444 493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.547568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38762 (local address 127.9.236.60:38141)
0929 19:14:34.547736 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.547738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.547744 (+ 6us) server_negotiation.cc:407] Connection header received
0929 19:14:34.547761 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.547761 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.547769 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.547824 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.548076 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.548757 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.549745 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.550074 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.550369 (+ 295us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.551864 (+ 1495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:34.551866 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:34.551866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:34.551872 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:34.553424 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.554585 (+ 1161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.554586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.554586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.554640 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.554838 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.554839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.554840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.555002 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:34.555064 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.555207 (+ 143us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.555294 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":115,"threads_started":1}
I20260929 19:14:34.555908 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.547344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43570 (local address 127.9.236.61:41435)
0929 19:14:34.547400 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.547402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.547529 (+ 127us) server_negotiation.cc:407] Connection header received
0929 19:14:34.547567 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.547568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.547578 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.547643 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.548134 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.548732 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.549788 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.550145 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.550192 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.552906 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:34.552908 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:34.552908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:34.552918 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:34.554504 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.554929 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.554931 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.554932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.555020 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.555208 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.555208 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.555209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.555340 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:34.555492 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.555730 (+ 238us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.555815 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:14:34.556847 488 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9b029c03701466f9cc8d5650dd9c0e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08803b1193542f5855a0f23f7fd086c" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38803 } } peers { permanent_uuid: "a21497ce397e4b27b594e197f24736a0" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } } peers { permanent_uuid: "e9b029c03701466f9cc8d5650dd9c0e4" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } }
I20260929 19:14:34.557014 488 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9b029c03701466f9cc8d5650dd9c0e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9b029c03701466f9cc8d5650dd9c0e4, State: Initialized, Role: FOLLOWER
I20260929 19:14:34.557157 488 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9b029c03701466f9cc8d5650dd9c0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08803b1193542f5855a0f23f7fd086c" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38803 } } peers { permanent_uuid: "a21497ce397e4b27b594e197f24736a0" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } } peers { permanent_uuid: "e9b029c03701466f9cc8d5650dd9c0e4" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } }
I20260929 19:14:34.557263 10160 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 19:14:34.557451 338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c: Bootstrap starting.
I20260929 19:14:34.557503 488 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9b029c03701466f9cc8d5650dd9c0e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:34.557499 495 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9b029c03701466f9cc8d5650dd9c0e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08803b1193542f5855a0f23f7fd086c" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38803 } } peers { permanent_uuid: "a21497ce397e4b27b594e197f24736a0" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } } peers { permanent_uuid: "e9b029c03701466f9cc8d5650dd9c0e4" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } } }
I20260929 19:14:34.557623 495 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9b029c03701466f9cc8d5650dd9c0e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:14:34.558130 338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:34.558460 338 log.cc:824] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:34.559368 338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c: No bootstrap required, opened a new log
I20260929 19:14:34.560528 338 raft_consensus.cc:348] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08803b1193542f5855a0f23f7fd086c" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38803 } } peers { permanent_uuid: "a21497ce397e4b27b594e197f24736a0" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } } peers { permanent_uuid: "e9b029c03701466f9cc8d5650dd9c0e4" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } }
I20260929 19:14:34.560636 338 raft_consensus.cc:729] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f08803b1193542f5855a0f23f7fd086c, State: Initialized, Role: FOLLOWER
I20260929 19:14:34.560686 338 consensus_queue.cc:260] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08803b1193542f5855a0f23f7fd086c" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38803 } } peers { permanent_uuid: "a21497ce397e4b27b594e197f24736a0" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } } peers { permanent_uuid: "e9b029c03701466f9cc8d5650dd9c0e4" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } }
I20260929 19:14:34.558226 312 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f08803b1193542f5855a0f23f7fd086c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45786
I20260929 19:14:34.560952 509 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08803b1193542f5855a0f23f7fd086c" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38803 } } peers { permanent_uuid: "a21497ce397e4b27b594e197f24736a0" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } } peers { permanent_uuid: "e9b029c03701466f9cc8d5650dd9c0e4" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } } }
I20260929 19:14:34.560953 338 sys_catalog.cc:564] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:34.561017 312 raft_consensus.cc:482] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 19:14:34.561182 312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:34.561092 509 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [sys.catalog]: This master's current role is: FOLLOWER
W20260929 19:14:34.559463 507 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e9b029c03701466f9cc8d5650dd9c0e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:14:34.561401 507 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:14:34.561996 312 raft_consensus.cc:504] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08803b1193542f5855a0f23f7fd086c" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38803 } } peers { permanent_uuid: "a21497ce397e4b27b594e197f24736a0" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } } peers { permanent_uuid: "e9b029c03701466f9cc8d5650dd9c0e4" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } }
I20260929 19:14:34.562255 312 leader_election.cc:288] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [CANDIDATE]: Term 1 election: Requested vote from peers a21497ce397e4b27b594e197f24736a0 (127.9.236.61:41435), e9b029c03701466f9cc8d5650dd9c0e4 (127.9.236.60:38141)
I20260929 19:14:34.562686 462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f08803b1193542f5855a0f23f7fd086c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e9b029c03701466f9cc8d5650dd9c0e4"
I20260929 19:14:34.562791 462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9b029c03701466f9cc8d5650dd9c0e4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:34.563630 462 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e9b029c03701466f9cc8d5650dd9c0e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f08803b1193542f5855a0f23f7fd086c in term 1.
I20260929 19:14:34.564604 32754 leader_election.cc:302] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e9b029c03701466f9cc8d5650dd9c0e4, f08803b1193542f5855a0f23f7fd086c; no voters:
I20260929 19:14:34.564970 509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 19:14:34.565213 509 raft_consensus.cc:686] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [term 1 LEADER]: Becoming Leader. State: Replica: f08803b1193542f5855a0f23f7fd086c, State: Running, Role: LEADER
I20260929 19:14:34.565320 509 consensus_queue.cc:237] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08803b1193542f5855a0f23f7fd086c" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38803 } } peers { permanent_uuid: "a21497ce397e4b27b594e197f24736a0" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } } peers { permanent_uuid: "e9b029c03701466f9cc8d5650dd9c0e4" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } }
I20260929 19:14:34.565817 509 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f08803b1193542f5855a0f23f7fd086c. Latest consensus state: current_term: 1 leader_uuid: "f08803b1193542f5855a0f23f7fd086c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08803b1193542f5855a0f23f7fd086c" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38803 } } peers { permanent_uuid: "a21497ce397e4b27b594e197f24736a0" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } } peers { permanent_uuid: "e9b029c03701466f9cc8d5650dd9c0e4" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } } }
I20260929 19:14:34.565888 509 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:34.564679 420 master_runner.cc:186] Error getting master registration for 127.9.236.61:41435: OK, instance_id { permanent_uuid: "a21497ce397e4b27b594e197f24736a0" instance_seqno: 1790709274353719 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 19:14:34.567144 420 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 19:14:34.566395 32738 master_runner.cc:186] Error getting master registration for 127.9.236.61:41435: OK, instance_id { permanent_uuid: "a21497ce397e4b27b594e197f24736a0" instance_seqno: 1790709274353719 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 19:14:34.569181 32738 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260929 19:14:34.569924 520 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 19:14:34.569973 520 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 19:14:34.570015 529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 19:14:34.570179 529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 19:14:34.571889 462 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e9b029c03701466f9cc8d5650dd9c0e4 [term 1 FOLLOWER]: Refusing update from remote peer f08803b1193542f5855a0f23f7fd086c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:14:34.572397 509 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9b029c03701466f9cc8d5650dd9c0e4" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:14:34.573659 495 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9b029c03701466f9cc8d5650dd9c0e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f08803b1193542f5855a0f23f7fd086c. Latest consensus state: current_term: 1 leader_uuid: "f08803b1193542f5855a0f23f7fd086c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08803b1193542f5855a0f23f7fd086c" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38803 } } peers { permanent_uuid: "a21497ce397e4b27b594e197f24736a0" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } } peers { permanent_uuid: "e9b029c03701466f9cc8d5650dd9c0e4" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } } }
I20260929 19:14:34.573753 495 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9b029c03701466f9cc8d5650dd9c0e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:14:34.575881 341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.564957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45798 (local address 127.9.236.62:38803)
0929 19:14:34.565017 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.565020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.566533 (+ 1513us) server_negotiation.cc:407] Connection header received
0929 19:14:34.566855 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.566856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.566870 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.566903 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.568571 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.569296 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.570520 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.570827 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.570889 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.572633 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:34.572635 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:34.572635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:34.572644 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:34.574329 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.574706 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.574707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.574708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.574759 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.574952 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.574953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.574953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.575093 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:34.575199 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.575682 (+ 483us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.575762 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 19:14:34.578922 509 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f08803b1193542f5855a0f23f7fd086c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08803b1193542f5855a0f23f7fd086c" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38803 } } peers { permanent_uuid: "a21497ce397e4b27b594e197f24736a0" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } } peers { permanent_uuid: "e9b029c03701466f9cc8d5650dd9c0e4" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } } }
I20260929 19:14:34.579020 509 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:34.579533 495 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9b029c03701466f9cc8d5650dd9c0e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f08803b1193542f5855a0f23f7fd086c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08803b1193542f5855a0f23f7fd086c" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38803 } } peers { permanent_uuid: "a21497ce397e4b27b594e197f24736a0" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } } peers { permanent_uuid: "e9b029c03701466f9cc8d5650dd9c0e4" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } } }
I20260929 19:14:34.579624 495 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9b029c03701466f9cc8d5650dd9c0e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:14:34.579877 532 mvcc.cc:205] Tried to move back new op lower bound from 7334745188643053568 to 7334745188620361728. Current Snapshot: MvccSnapshot[applied={T|T < 7334745188643053568}]
I20260929 19:14:34.581171 533 mvcc.cc:205] Tried to move back new op lower bound from 7334745188643053568 to 7334745188620361728. Current Snapshot: MvccSnapshot[applied={T|T < 7334745188643053568}]
I20260929 19:14:34.583585 493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.569250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38774 (local address 127.9.236.60:38141)
0929 19:14:34.569290 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.569292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.576063 (+ 6771us) server_negotiation.cc:407] Connection header received
0929 19:14:34.576165 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.576167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.576180 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.576211 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.576997 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.577737 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.578722 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.579073 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.579113 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.580547 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:34.580549 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:34.580549 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:34.580563 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:34.582330 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.582692 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.582693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.582694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.582744 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.582940 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.582941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.582941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.583075 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:34.583174 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.583382 (+ 208us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.583467 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 19:14:34.585181 529 catalog_manager.cc:1347] Generated new cluster ID: 113a5c230e1845efa972137bf953edc7
I20260929 19:14:34.585242 529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 19:14:34.591144 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.565385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43574 (local address 127.9.236.61:41435)
0929 19:14:34.565424 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.565426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.570472 (+ 5046us) server_negotiation.cc:407] Connection header received
0929 19:14:34.580953 (+ 10481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.580956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.580976 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.581008 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.584528 (+ 3520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.585264 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.586351 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.586682 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.586727 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.588265 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:34.588267 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:34.588268 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:34.588275 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:34.589867 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.590259 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.590260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.590260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.590314 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.590518 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.590519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.590519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.590676 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:34.590750 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.590961 (+ 211us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.591045 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260929 19:14:34.591747 413 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } attempt: 3
I20260929 19:14:34.592911 529 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 19:14:34.594257 529 catalog_manager.cc:1504] Loading token signing keys...
Sep 29 19:14:34 dist-test-slave-wftq krb5kdc[32725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790709274, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.236.61@KRBTEST.COM for kudu/127.9.236.60@KRBTEST.COM
I20260929 19:14:34.601205 493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.592165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38780 (local address 127.9.236.60:38141)
0929 19:14:34.592206 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.592207 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.592216 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:14:34.592241 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.592242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.592251 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.592297 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.592670 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.593513 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.594613 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.594922 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.594978 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.597970 (+ 2992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 19:14:34.597972 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 19:14:34.597973 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 19:14:34.597980 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 19:14:34.599727 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.600157 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.600159 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.600159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.600227 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 19:14:34.600439 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 19:14:34.600439 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 19:14:34.600440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 19:14:34.600587 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 19:14:34.600644 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.600920 (+ 276us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.601057 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260929 19:14:34.602324 415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.592063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.236.60:38141 (local address 127.0.0.1:38780)
0929 19:14:34.592146 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0929 19:14:34.592157 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 19:14:34.592197 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 19:14:34.592345 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 19:14:34.592346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 19:14:34.592538 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0929 19:14:34.592591 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:34.592593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.593602 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.593603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:34.594432 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 19:14:34.594434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.595063 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.595064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 19:14:34.595134 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.595621 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 19:14:34.595626 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 19:14:34.597861 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 19:14:34.599840 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:34.599841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:34.599842 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:34.600066 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:34.600300 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 19:14:34.600301 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 19:14:34.600301 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0929 19:14:34.600370 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 19:14:34.600692 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 19:14:34.600693 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 19:14:34.600762 (+ 69us) client_negotiation.cc:769] Sending connection context
0929 19:14:34.600827 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 19:14:34.602212 (+ 1385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260929 19:14:34.604212 413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a21497ce397e4b27b594e197f24736a0: Bootstrap starting.
I20260929 19:14:34.604429 529 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c: Generated new TSK 0
I20260929 19:14:34.604859 529 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 19:14:34.604967 413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a21497ce397e4b27b594e197f24736a0: Neither blocks nor log segments found. Creating new log.
I20260929 19:14:34.605304 413 log.cc:824] T 00000000000000000000000000000000 P a21497ce397e4b27b594e197f24736a0: Log is configured to *not* fsync() on all Append() calls
I20260929 19:14:34.606398 413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a21497ce397e4b27b594e197f24736a0: No bootstrap required, opened a new log
I20260929 19:14:34.608744 413 raft_consensus.cc:348] T 00000000000000000000000000000000 P a21497ce397e4b27b594e197f24736a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08803b1193542f5855a0f23f7fd086c" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38803 } } peers { permanent_uuid: "a21497ce397e4b27b594e197f24736a0" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } } peers { permanent_uuid: "e9b029c03701466f9cc8d5650dd9c0e4" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } }
I20260929 19:14:34.608924 413 raft_consensus.cc:729] T 00000000000000000000000000000000 P a21497ce397e4b27b594e197f24736a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a21497ce397e4b27b594e197f24736a0, State: Initialized, Role: FOLLOWER
I20260929 19:14:34.609032 413 consensus_queue.cc:260] T 00000000000000000000000000000000 P a21497ce397e4b27b594e197f24736a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08803b1193542f5855a0f23f7fd086c" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38803 } } peers { permanent_uuid: "a21497ce397e4b27b594e197f24736a0" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } } peers { permanent_uuid: "e9b029c03701466f9cc8d5650dd9c0e4" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } }
I20260929 19:14:34.609627 386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a21497ce397e4b27b594e197f24736a0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 19:14:34.610733 386 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a21497ce397e4b27b594e197f24736a0 [term 1 FOLLOWER]: Refusing update from remote peer f08803b1193542f5855a0f23f7fd086c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 19:14:34.611137 526 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a21497ce397e4b27b594e197f24736a0" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 19:14:34.609582 387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f08803b1193542f5855a0f23f7fd086c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a21497ce397e4b27b594e197f24736a0"
I20260929 19:14:34.609526 413 sys_catalog.cc:564] T 00000000000000000000000000000000 P a21497ce397e4b27b594e197f24736a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 19:14:34.609519 539 sys_catalog.cc:455] T 00000000000000000000000000000000 P a21497ce397e4b27b594e197f24736a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08803b1193542f5855a0f23f7fd086c" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38803 } } peers { permanent_uuid: "a21497ce397e4b27b594e197f24736a0" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } } peers { permanent_uuid: "e9b029c03701466f9cc8d5650dd9c0e4" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } } }
I20260929 19:14:34.614310 539 sys_catalog.cc:458] T 00000000000000000000000000000000 P a21497ce397e4b27b594e197f24736a0 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:14:34.614418 539 sys_catalog.cc:455] T 00000000000000000000000000000000 P a21497ce397e4b27b594e197f24736a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f08803b1193542f5855a0f23f7fd086c. Latest consensus state: current_term: 1 leader_uuid: "f08803b1193542f5855a0f23f7fd086c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08803b1193542f5855a0f23f7fd086c" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38803 } } peers { permanent_uuid: "a21497ce397e4b27b594e197f24736a0" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } } peers { permanent_uuid: "e9b029c03701466f9cc8d5650dd9c0e4" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } } }
I20260929 19:14:34.614518 539 sys_catalog.cc:458] T 00000000000000000000000000000000 P a21497ce397e4b27b594e197f24736a0 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:14:34.614583 539 sys_catalog.cc:455] T 00000000000000000000000000000000 P a21497ce397e4b27b594e197f24736a0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f08803b1193542f5855a0f23f7fd086c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08803b1193542f5855a0f23f7fd086c" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38803 } } peers { permanent_uuid: "a21497ce397e4b27b594e197f24736a0" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } } peers { permanent_uuid: "e9b029c03701466f9cc8d5650dd9c0e4" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } } }
I20260929 19:14:34.614665 539 sys_catalog.cc:458] T 00000000000000000000000000000000 P a21497ce397e4b27b594e197f24736a0 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 19:14:34.618089 387 raft_consensus.cc:2396] T 00000000000000000000000000000000 P a21497ce397e4b27b594e197f24736a0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f08803b1193542f5855a0f23f7fd086c for term 1 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260929 19:14:34.619472 541 mvcc.cc:205] Tried to move back new op lower bound from 7334745188773167104 to 7334745188620361728. Current Snapshot: MvccSnapshot[applied={T|T < 7334745188643053568}]
I20260929 19:14:34.620466 554 catalog_manager.cc:1259] Loaded cluster ID: 113a5c230e1845efa972137bf953edc7
I20260929 19:14:34.620518 554 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a21497ce397e4b27b594e197f24736a0: loading cluster ID for follower catalog manager: success
I20260929 19:14:34.621775 554 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a21497ce397e4b27b594e197f24736a0: acquiring CA information for follower catalog manager: success
I20260929 19:14:34.622171 554 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a21497ce397e4b27b594e197f24736a0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 19:14:34.640232 341 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:34.638662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51508 (local address 127.9.236.62:38803)
0929 19:14:34.638721 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.638723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.639464 (+ 741us) server_negotiation.cc:407] Connection header received
0929 19:14:34.639578 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.639579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.639593 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.639625 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.640138 (+ 513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51508: BlockingRecv error: recv got EOF from 127.0.0.1:51508 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260929 19:14:34.641194 417 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:34.639164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53168 (local address 127.9.236.61:41435)
0929 19:14:34.639210 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.639211 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.640618 (+ 1407us) server_negotiation.cc:407] Connection header received
0929 19:14:34.640713 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.640714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.640726 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.640755 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.641075 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53168: BlockingRecv error: recv got EOF from 127.0.0.1:53168 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260929 19:14:34.642216 493 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:34.641557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53956 (local address 127.9.236.60:38141)
0929 19:14:34.641605 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.641607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.641722 (+ 115us) server_negotiation.cc:407] Connection header received
0929 19:14:34.641808 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.641809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.641820 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.641846 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.642149 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53956: BlockingRecv error: recv got EOF from 127.0.0.1:53956 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260929 19:14:34.647543 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.643905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53180 (local address 127.9.236.61:41435)
0929 19:14:34.643955 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.643957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.644156 (+ 199us) server_negotiation.cc:407] Connection header received
0929 19:14:34.644259 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.644260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.644317 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.644369 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:34.644576 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.645430 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.646573 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.646841 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.646883 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.647124 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:34.647280 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:34.647335 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.647437 (+ 102us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.647461 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260929 19:14:34.651120 493 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:34.650321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53966 (local address 127.9.236.60:38141)
0929 19:14:34.650389 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.650391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.650513 (+ 122us) server_negotiation.cc:407] Connection header received
0929 19:14:34.650619 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.650620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.650641 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.650668 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.651058 (+ 390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53966: BlockingRecv error: recv got EOF from 127.0.0.1:53966 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 19:14:34.656667 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.653502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53186 (local address 127.9.236.61:41435)
0929 19:14:34.653540 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.653542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.653775 (+ 233us) server_negotiation.cc:407] Connection header received
0929 19:14:34.653862 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.653863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.653873 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.653898 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:34.654079 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.654787 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.655831 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.656128 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.656175 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.656380 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:34.656418 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:34.656446 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.656553 (+ 107us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.656579 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260929 19:14:34.660527 493 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:34.659561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53982 (local address 127.9.236.60:38141)
0929 19:14:34.659612 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.659614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.659909 (+ 295us) server_negotiation.cc:407] Connection header received
0929 19:14:34.660028 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.660030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.660043 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.660070 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.660458 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53982: BlockingRecv error: recv got EOF from 127.0.0.1:53982 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 19:14:34.667160 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.663756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53190 (local address 127.9.236.61:41435)
0929 19:14:34.663800 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.663802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.664079 (+ 277us) server_negotiation.cc:407] Connection header received
0929 19:14:34.664173 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.664175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.664189 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.664217 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:34.664389 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.665246 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.666313 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.666606 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.666651 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.666855 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:34.666895 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:34.666922 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.667042 (+ 120us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.667070 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260929 19:14:34.675707 493 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:34.674721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53998 (local address 127.9.236.60:38141)
0929 19:14:34.674774 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.674776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.674910 (+ 134us) server_negotiation.cc:407] Connection header received
0929 19:14:34.675060 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.675061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.675075 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.675108 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.675630 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53998: BlockingRecv error: recv got EOF from 127.0.0.1:53998 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260929 19:14:34.685009 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.681387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53204 (local address 127.9.236.61:41435)
0929 19:14:34.681441 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.681443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.681784 (+ 341us) server_negotiation.cc:407] Connection header received
0929 19:14:34.681892 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.681893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.681906 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.681948 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:34.682156 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.682928 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.684134 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.684425 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.684484 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.684699 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:34.684742 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:34.684769 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.684868 (+ 99us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.684892 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260929 19:14:34.688851 493 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:34.687857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54008 (local address 127.9.236.60:38141)
0929 19:14:34.687912 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.687914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.688203 (+ 289us) server_negotiation.cc:407] Connection header received
0929 19:14:34.688305 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.688306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.688320 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.688351 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.688773 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54008: BlockingRecv error: recv got EOF from 127.0.0.1:54008 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260929 19:14:34.701967 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.698445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53210 (local address 127.9.236.61:41435)
0929 19:14:34.698502 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.698505 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.698812 (+ 307us) server_negotiation.cc:407] Connection header received
0929 19:14:34.698919 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.698920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.698932 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.698975 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:34.699176 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.699967 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.701128 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.701419 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.701468 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.701663 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:34.701702 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:34.701728 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.701824 (+ 96us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.701848 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260929 19:14:34.705713 493 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:34.704880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54018 (local address 127.9.236.60:38141)
0929 19:14:34.704949 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.704951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.705050 (+ 99us) server_negotiation.cc:407] Connection header received
0929 19:14:34.705188 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.705189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.705203 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.705232 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.705639 (+ 407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54018: BlockingRecv error: recv got EOF from 127.0.0.1:54018 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 19:14:34.727046 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.723514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53220 (local address 127.9.236.61:41435)
0929 19:14:34.723570 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.723572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.723915 (+ 343us) server_negotiation.cc:407] Connection header received
0929 19:14:34.724037 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.724039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.724051 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.724081 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:34.724281 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.725071 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.726226 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.726520 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.726562 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.726756 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:34.726794 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:34.726822 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.726917 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.726953 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260929 19:14:34.730620 493 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:34.729711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54020 (local address 127.9.236.60:38141)
0929 19:14:34.729758 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.729760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.729933 (+ 173us) server_negotiation.cc:407] Connection header received
0929 19:14:34.730035 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.730036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.730062 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.730090 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.730556 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54020: BlockingRecv error: recv got EOF from 127.0.0.1:54020 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 19:14:34.766446 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.763967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53236 (local address 127.9.236.61:41435)
0929 19:14:34.764022 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.764024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.764246 (+ 222us) server_negotiation.cc:407] Connection header received
0929 19:14:34.764315 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.764316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.764328 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.764386 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:34.764502 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.765020 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.765826 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.766021 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.766077 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.766203 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:34.766238 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:34.766276 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.766329 (+ 53us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.766347 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260929 19:14:34.768889 493 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:34.768332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54022 (local address 127.9.236.60:38141)
0929 19:14:34.768382 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.768384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.768392 (+ 8us) server_negotiation.cc:407] Connection header received
0929 19:14:34.768462 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.768463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.768476 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.768518 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.768830 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54022: BlockingRecv error: recv got EOF from 127.0.0.1:54022 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 19:14:34.836861 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.834430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53242 (local address 127.9.236.61:41435)
0929 19:14:34.834509 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.834513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.834627 (+ 114us) server_negotiation.cc:407] Connection header received
0929 19:14:34.834713 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.834715 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.834729 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.834779 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:34.834949 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.835477 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.836239 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.836436 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.836491 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.836619 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:34.836651 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:34.836693 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.836746 (+ 53us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.836768 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260929 19:14:34.839387 493 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:34.838785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54032 (local address 127.9.236.60:38141)
0929 19:14:34.838848 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.838850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.838861 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:14:34.838927 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.838928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.838939 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.838986 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.839326 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54032: BlockingRecv error: recv got EOF from 127.0.0.1:54032 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260929 19:14:34.971056 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:34.968577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53256 (local address 127.9.236.61:41435)
0929 19:14:34.968645 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.968648 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.968663 (+ 15us) server_negotiation.cc:407] Connection header received
0929 19:14:34.968728 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.968729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.968741 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.968835 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:34.968950 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.969533 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.970360 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:34.970556 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:34.970615 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:34.970771 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:34.970808 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:34.970845 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:34.970962 (+ 117us) server_negotiation.cc:299] Negotiation successful
0929 19:14:34.970980 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260929 19:14:34.973489 493 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:34.972899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54040 (local address 127.9.236.60:38141)
0929 19:14:34.972967 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:34.972969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:34.972997 (+ 28us) server_negotiation.cc:407] Connection header received
0929 19:14:34.973024 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:34.973026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:34.973037 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:34.973080 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:34.973433 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54040: BlockingRecv error: recv got EOF from 127.0.0.1:54040 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 19:14:35.118623 526 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f08803b1193542f5855a0f23f7fd086c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08803b1193542f5855a0f23f7fd086c" member_type: VOTER last_known_addr { host: "127.9.236.62" port: 38803 } } peers { permanent_uuid: "a21497ce397e4b27b594e197f24736a0" member_type: VOTER last_known_addr { host: "127.9.236.61" port: 41435 } } peers { permanent_uuid: "e9b029c03701466f9cc8d5650dd9c0e4" member_type: VOTER last_known_addr { host: "127.9.236.60" port: 38141 } } }
I20260929 19:14:35.118754 526 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08803b1193542f5855a0f23f7fd086c [sys.catalog]: This master's current role is: LEADER
I20260929 19:14:35.233347 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:35.230745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53272 (local address 127.9.236.61:41435)
0929 19:14:35.230844 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:35.230846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:35.230857 (+ 11us) server_negotiation.cc:407] Connection header received
0929 19:14:35.230890 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:35.230891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:35.230902 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:35.230991 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:35.231172 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:35.231726 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:35.232605 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:35.232845 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:35.232912 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:35.233022 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:35.233053 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:35.233091 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:35.233247 (+ 156us) server_negotiation.cc:299] Negotiation successful
0929 19:14:35.233269 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260929 19:14:35.235803 493 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 19:14:35.235228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54048 (local address 127.9.236.60:38141)
0929 19:14:35.235315 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:35.235317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:35.235327 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:14:35.235354 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:35.235355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:35.235367 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:35.235415 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 19:14:35.235742 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54048: BlockingRecv error: recv got EOF from 127.0.0.1:54048 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 19:14:35.561836 507 catalog_manager.cc:1259] Loaded cluster ID: 113a5c230e1845efa972137bf953edc7
I20260929 19:14:35.561921 507 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e9b029c03701466f9cc8d5650dd9c0e4: loading cluster ID for follower catalog manager: success
I20260929 19:14:35.562886 507 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e9b029c03701466f9cc8d5650dd9c0e4: acquiring CA information for follower catalog manager: success
I20260929 19:14:35.563202 507 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e9b029c03701466f9cc8d5650dd9c0e4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 19:14:35.751514 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:35.749142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53280 (local address 127.9.236.61:41435)
0929 19:14:35.749316 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:35.749319 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:35.749329 (+ 10us) server_negotiation.cc:407] Connection header received
0929 19:14:35.749364 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:35.749365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:35.749377 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:35.749438 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:35.749560 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:35.750104 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:35.750869 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:35.751075 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:35.751133 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:35.751248 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:35.751288 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:35.751329 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:35.751382 (+ 53us) server_negotiation.cc:299] Negotiation successful
0929 19:14:35.751409 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":78,"threads_started":1}
I20260929 19:14:35.755918 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:35.753392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54056 (local address 127.9.236.60:38141)
0929 19:14:35.753507 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:35.753509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:35.753518 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:14:35.753582 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:35.753582 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:35.753644 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:35.753732 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:35.753904 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:35.754400 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:35.755188 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:35.755389 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:35.755444 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:35.755562 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:35.755685 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:35.755743 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:35.755761 (+ 18us) server_negotiation.cc:299] Negotiation successful
0929 19:14:35.755820 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
I20260929 19:14:35.760555 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:35.757823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54062 (local address 127.9.236.60:38141)
0929 19:14:35.757884 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:35.757886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:35.757961 (+ 75us) server_negotiation.cc:407] Connection header received
0929 19:14:35.758047 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:35.758048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:35.758056 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:35.758097 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:35.758243 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:35.758786 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:35.759732 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:35.760062 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:35.760134 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:35.760308 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:35.760346 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:35.760390 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:35.760447 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 19:14:35.760474 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 19:14:35.760957 674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:35.757831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53296 (local address 127.9.236.61:41435)
0929 19:14:35.757869 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:35.757871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:35.758067 (+ 196us) server_negotiation.cc:407] Connection header received
0929 19:14:35.758142 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:35.758143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:35.758156 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:35.758205 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:35.758413 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:35.759168 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:35.760160 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:35.760436 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:35.760493 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:35.760671 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:35.760701 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:35.760727 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:35.760854 (+ 127us) server_negotiation.cc:299] Negotiation successful
0929 19:14:35.760881 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260929 19:14:35.761140 688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 19:14:35.757699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51510 (local address 127.9.236.62:38803)
0929 19:14:35.757961 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0929 19:14:35.757963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 19:14:35.757972 (+ 9us) server_negotiation.cc:407] Connection header received
0929 19:14:35.758031 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 19:14:35.758032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 19:14:35.758091 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 19:14:35.758167 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 19:14:35.758318 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:35.759089 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:35.760011 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 19:14:35.760291 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 19:14:35.760350 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 19:14:35.760582 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 19:14:35.760710 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 19:14:35.760822 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 19:14:35.760997 (+ 175us) server_negotiation.cc:299] Negotiation successful
0929 19:14:35.761031 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":85,"threads_started":1}
I20260929 19:14:35.768939 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 32738
I20260929 19:14:35.773851 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 345
I20260929 19:14:35.778918 10160 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0UA8iT/build/release/bin/kudu with pid 420
2026-09-29T19:14:35Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2785 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5521 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (267265 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST